Modeling Booklet Effects for Nonequivalent Group Designs in Large-Scale Assessment

Autor/inn/en	Hecht, Martin; Weirich, Sebastian; Siegle, Thilo; Frey, Andreas
Titel	Modeling Booklet Effects for Nonequivalent Group Designs in Large-Scale Assessment
Quelle	In: Educational and Psychological Measurement, 75 (2015) 4, S.568-584 (17 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/0013164414554219
Schlagwörter	Measurement; Item Sampling; Statistical Analysis; Models; Item Response Theory; Grade 9; Grade 10; National Competency Tests; Science Tests; Foreign Countries; Germany + Suchen Sie Ihr Suchwort? Messverfahren; Statistische Analyse; Analogiemodell; Item-Response-Theorie; School year 09; 9. Schuljahr; Schuljahr 09; Ausland; Deutschland
Abstract	Multiple matrix designs are commonly used in large-scale assessments to distribute test items to students. These designs comprise several booklets, each containing a subset of the complete item pool. Besides reducing the test burden of individual students, using various booklets allows aligning the difficulty of the presented items to the assumed performance level of examined subgroups. While this may improve measurement precision and students' test-taking motivation, using several booklets might influence response behavior and thus constitute a potential source of unwanted variation. To provide guidance to identify and model booklet effects, this study presents statistical models accounting for booklet effects and applies these models in a large-scale assessment setting. Three models are derived from the Rasch model employing the generalized linear mixed models framework. The models were applied to data from a national educational standards assessment study for scientific competence. A total of 1,021 items were compiled to 74 booklets distributed to a sample of 9,044 students of Grades 9 and 10. The results revealed a small but nonnegligible booklet effect. For further large-scale assessment studies, it is recommended to examine whether booklet effects occur and to adequately account for them in the subsequent analyses where necessary. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01