Modeling test context effects in longitudinal achievement data: Examining position effects in the longitudinal German PISA 2012 assessment.

Autor/inn/en	Nagy, Gabriel; Lüdtke, Oliver; Köller, Olaf
Titel	Modeling test context effects in longitudinal achievement data: Examining position effects in the longitudinal German PISA 2012 assessment.
Quelle	In: Psychological test and assessment modeling, 58 (2016) 4, S. 641-670 PDF als Volltext kostenfreie Datei Link als defekt meldenVerfügbarkeit
Sprache	englisch
Dokumenttyp	online; Zeitschriftenaufsatz
ISSN	2190-0493
URN	urn:nbn:de:0111-pedocs-128040
Schlagwörter	Evaluation; Längsschnittuntersuchung; Testkonstruktion; Psychometrie; Test; Deutschland; Längsschnittuntersuchung; Modell; Qualität; Bildungsstandards; Test; Evaluation; Testkonstruktion; Leistungsmessung; Psychometrie; Wirkung; Item; Schuljahr 09; Schülerleistung; Item; Leistungsmessung; Modell; Qualität; Wirkung; PISA (Programme for International Student Assessment); Deutschland
Abstract	Position effects (PE) in school achievement tests are a specific kind of test context effects (TCEs) that refer to the phenomenon of items becoming more difficult, the later they are positioned in a test. Up until today, PEs have been investigated mainly in cross-sectional settings; this means that little is known about how the size of PEs changes when retesting students. In the present article, we investigate TCEs in the longitudinal extension of the PISA 2012 assessment in Germany. To this end, we propose an extension of the two-dimensional one-parameter item response model, with one dimension per measurement occasion, that includes the effects of booklets (i.e., test forms) on item clusters (i.e., item bundles) that are allowed to vary between assessment occasions and groups (school types). Results indicate that the TCEs uncovered in all domains tested (mathematics, science, and reading) are closely in line with PEs, with reading being most strongly affected, and mathematics being least affected. The size of PEs increased in the second assessment, although the domains were differently affected. This pattern of effects was more pronounced in nonacademic school types. Finally, estimates of average achievement gains appeared to be underestimated by IRT models that neglected TCEs, with differences being largest in domains most strongly affected by PEs (i.e., science and reading).
Erfasst von	DIPF \| Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main
Update	2017/2