How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex Student Construct

Autor/inn/en	Sung, Shannon H.; Li, Chenglu; Chen, Guanhua; Huang, Xudong; Xie, Charles; Massicotte, Joyce; Shen, Ji
Titel	How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex Student Construct
Quelle	In: Journal of Science Education and Technology, 30 (2021) 2, S.210-226 (17 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Sung, Shannon H.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1059-0145
DOI	10.1007/s10956-020-09856-2
Schlagwörter	Observation; Artificial Intelligence; Knowledge Representation; Grade 9; High School Students; Thermodynamics; Science Instruction; Inquiry; Active Learning; Automation; Student Evaluation; Classification; Computer Uses in Education; Secondary School Science + Suchen Sie Ihr Suchwort? Beobachtung; Künstliche Intelligenz; Wissensrepräsentation; School year 09; 9. Schuljahr; Schuljahr 09; High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Thermodynamik; Teaching of science; Science education; Natural sciences Lessons; Naturwissenschaftlicher Unterricht; Aktives Lernen; Schulnote; Studentische Bewertung; Classification system; Klassifikation; Klassifikationssystem; Computernutzung
Abstract	In this paper, we demonstrate how machine learning could be used to quickly assess a student's multimodal representational thinking. Multimodal representational thinking is the complex construct that encodes how students form conceptual, perceptual, graphical, or mathematical symbols in their mind. The augmented reality (AR) technology is adopted to diversify student's representations. The AR technology utilized a low-cost, high-resolution thermal camera attached to a smartphone which allows students to explore the unseen world of thermodynamics. Ninth-grade students (N = 314) engaged in a prediction-observation-explanation (POE) inquiry cycle scaffolded to leverage the augmented observation provided by the aforementioned device. The objective is to investigate how machine learning could expedite the automated assessment of multimodal representational thinking of heat energy. Two automated text classification methods were adopted to decode different mental representations students used to explain their haptic perception, thermal imaging, and graph data collected in the lab. Since current automated assessment in science education rarely considers multilabel classification, we resorted to the help of the state-of-the-art deep learning technique-bidirectional encoder representations from transformers (BERT). The BERT model classified open-ended responses into appropriate categories with higher precision than the traditional machine learning method. The satisfactory accuracy of deep learning in assigning multiple labels is revolutionary in processing qualitative data. The complex student construct, such as multimodal representational thinking, is rarely mutually exclusive. The study avails a convenient technique to analyze qualitative data that does not satisfy the mutual-exclusiveness assumption. Implications and future studies are discussed. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01