AI scoring for international large-scale assessments using a deep learning model and multilingual data.

Autor/inn/en	Okubo, Tomoya; Houlden, Wayne; Montuoro, Paul; Reinertsen, Nate; Tse, Chi Sum; Bastianic, Tanja
Titel	AI scoring for international large-scale assessments using a deep learning model and multilingual data.
Quelle	Paris: OECD Publishing (2023), 34 S. PDF als Volltext Link als defekt meldenVerfügbarkeit
Reihe	OECD education working papers. 286
Beigaben	Literaturangaben; Tabellen; Abbildungen
Sprache	englisch
Dokumenttyp	online; Monographie
DOI	10.1787/9918e1fb-en
Schlagwörter	Künstliche Intelligenz; Bewertung; Künstliche Intelligenz; Mehrsprachigkeit; Daten; Methodik; Mehrsprachigkeit; Arbeitspapier; Bewertung; Daten; OECD-Studie; Studie; Studie; PISA (Programme for International Student Assessment)
Abstract	Artificial Intelligence (AI) scoring for constructed-response items, using recent advancements in multilingual, deep learning techniques utilising models pre-trained with a massive multilingual text corpus, is examined using international large-scale assessment data. Historical student responses to Reading and Science literacy cognitive items developed under the PISA analytical framework are used as training data for deep learning together with multilingual data to construct an AI model. The trained AI models are then used to score and the results compared with human-scored data. The score distributions estimated based on the AI-scored data and the human-scored data are highly consistent with each other; furthermore, even item-level psychometric properties of the majority of items showed high levels of agreement, although a few items showed discrepancies. This study demonstrates a practical procedure for using a multilingual data approach, and this new AI-scoring methodology reached a practical level of quality, even in the context of an international large-scale assessment.
Erfasst von	DIPF \| Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main (extern)
Update	2023/1