The Rating and Matching Item-Objective Alignment Methods

Autor/inn/en	D'Agostino, Jerome V.; Welsh, Megan E.; Cimetta, Adriana D.; Falco, Lia D.; Smith, Shannon; VanWinkle, Waverely Hester; Powers, Sonya J.
Titel	The Rating and Matching Item-Objective Alignment Methods
Quelle	In: Applied Measurement in Education, 21 (2008) 1, S.1-21 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0895-7347
Schlagwörter	Test Items; Student Evaluation; State Standards; Evaluation Methods; Secondary School Mathematics; Academic Standards; High School Students; Mathematics Teachers; Graduate Students; Scores; Correlation; Measurement Techniques; Error Patterns; Judges; Arizona + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Schulnote; Studentische Bewertung; High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Mathematics; Teacher; Teachers; Mathematik; Lehrer; Lehrerin; Lehrende; Graduate Study; Aufbaustudium; Graduiertenstudium; Hauptstudium; Korrelation; Messtechnik; Fehlertyp; Judge; Richter
Abstract	Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two methods and on the item and objective characteristics that influence judges' decisions. We randomly assigned judges to either rate item-objective links or match items to objectives while reviewing the 2004 Arizona high school mathematics standards and assessment. Across items we found moderate convergence between methods, and we detected apparent reasons for divergently scored items. We also found that judges relied on item and objective content and intellectual skill features to render decisions. Based on our evidence, we contend that a thorough alignment analysis would involve judges using both rating and matching, while focusing on both content and intellectual skill. The findings have important implications for states when examining the alignment between their standards and assessments. (Contains 2 tables and 3 figures.) (Author).
Anmerkungen	Lawrence Erlbaum. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals/default.html
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10