Effects of design properties on parameter estimation in large-scale assessments.

Autor/inn/en	Hecht, Martin; Weirich, Sebastian; Siegle, Thilo; Frey, Andreas
Titel	Effects of design properties on parameter estimation in large-scale assessments.
Quelle	In: Educational and psychological measurement, 75 (2015) 6, S. 1021-1044Infoseite zur Zeitschrift PDF als Volltext Link als defekt meldenVerfügbarkeit
Zusatzinformation	Forschungsdaten, Studiendetails und Erhebungsinstrumente
Sprache	englisch
Dokumenttyp	online; gedruckt; Zeitschriftenaufsatz
ISSN	0013-1644; 1552-3888
DOI	10.1177/0013164415573311
Schlagwörter	Rasch-Modell; Bildungsforschung; Forschungsdesign; Vergleichsuntersuchung; Itemanalyse; Psychologische Messung; Test
Abstract	The selection of an appropriate booklet design is an important element of large-scale assessments of student achievement. Two design properties that are typically optimized are the balance with respect to the positions the items are presented and with respect to the mutual occurrence of pairs of items in the same booklet. The purpose of this study is to investigate the effects of these two design properties on bias and root mean square error of item parameter estimates from the Rasch model. First, position effects are estimated using data from a large-scale assessment study measuring the competencies of 19,107 ninth graders in science. These results were then used for a simulation study with 1,540 booklet designs with systematically varied position balance and cluster pair balance. The simulation results showed a small effect of position balancing on bias and root mean square error of the item parameter estimates while the cluster pair balance was ignorable. This null effect is actually good news for test designers since it allows for deliberately reducing the degree of cluster pair balance without negative effects on item parameter estimates. However, it is recommended to try to achieve a high position balance when designing large-scale assessment studies. (Verlag).
Erfasst von	Externer Selbsteintrag
Update	2016/2