Gauging Uncertainty in Test-to-Curriculum Alignment Indices

Autor/inn/en	Traynor, Anne; Li, Tingxuan; Zhou, Shuqi
Titel	Gauging Uncertainty in Test-to-Curriculum Alignment Indices
Quelle	In: Applied Measurement in Education, 33 (2020) 2, S.141-158 (18 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Traynor, Anne)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0895-7347
DOI	10.1080/08957347.2020.1732387
Schlagwörter	Alignment (Education); Achievement Tests; Curriculum; Error of Measurement; Expertise; Elementary Secondary Education; Test Items + Suchen Sie Ihr Suchwort? Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Curricula; Lehrplan; Rahmenplan; Messfehler; Expert appraisal; Test content; Testaufgabe
Abstract	During the development of large-scale school achievement tests, panels of independent subject-matter experts use systematic judgmental methods to rate the correspondence between a given test's items and performance objective statements. The individual experts' ratings may then be used to compute summary indices to quantify the match between a given test and its target item domain. The magnitude of alignment index variability across experts within a panel, and randomly-sampled panels, is largely unknown, however. Using rater-by-item data from alignment reviews of 14 US states' achievement tests, we examine observed distributions and estimate standard errors for three alignment indices developed by Webb. Our results suggest that alignment decisions based on the recommended criterion for the balance-of-representation index may often be uncertain, and that the criterion for the depth-of-knowledge consistency index should perhaps be reconsidered. We also examine current recommendations about the number of expert panelists required to compute these alignment indices. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01