PISA reading. Mode effects unveiled in short text responses.

Autor/inn/en	Zehner, Fabian; Kroehne, Ulf; Hahnel, Carolin; Goldhammer, Frank
Titel	PISA reading. Mode effects unveiled in short text responses.
Quelle	In: Psychological test and assessment modeling, 62 (2020) 1, S. 85-105 PDF als Volltext kostenfreie Datei (1); PDF als Volltext kostenfreie Datei (inhaltlich überarbeitete Version) Link als defekt meldenVerfügbarkeit
Sprache	englisch
Dokumenttyp	online; Zeitschriftenaufsatz
ISSN	2190-0493; 2190-0507
URN	urn:nbn:de:0111-pedocs-203542
Schlagwörter	Methodenwechsel; Sekundäranalyse; Leistungsbeurteilung; Schülerleistung; Text; Datenanalyse; Computerunterstütztes Verfahren; Papier; Antwort; Effekt; Inhalt; Quantität; Wirkungsforschung; Bleistift; Information; PISA (Programme for International Student Assessment); Deutschland
Abstract	Educational largescale assessments risk their temporal comparability when shifting from paperto computerbased assessment. A recent study showed how text responses have altered alongside PISA's mode change, indicating mode effects. Uncertainty remained, however, because it compared students from 2012 and 2015. We aimed at reproducing the findings in an experimental setting, in which n = 836 students answered PISA reading questions on computer, paper, or both. Text response features for information quantity and relevance were extracted automatically. Results show a comprehensive recovery of findings. Students incorporated more information into their text responses on computer than on paper, with some items being more affected than others. Regarding information relevance, we found less mode effect variance across items than the original study. Hints for a relationship between mode effect and gender across items could be reproduced. The study demonstrates the stability of linguistic feature extraction from text responses. (DIPF/Orig.).
Erfasst von	DIPF \| Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main
Update	2021/3