Effects of Assigning Raters to Items

Autor/inn/en	Sykes, Robert C.; Ito, Kyoko; Wang, Zhen
Titel	Effects of Assigning Raters to Items
Quelle	In: Educational Measurement: Issues and Practice, 27 (2008) 1, S.47-55 (9 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/j.1745-3992.2008.00114.x
Schlagwörter	Test Items; Mathematics Tests; Reading Tests; Scoring; Error of Measurement; Interrater Reliability; Measurement Techniques; Evaluators; Personnel Selection; Bias; Factor Analysis + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Lesetest; Bewertung; Messfehler; Interrater-Reliabilität; Messtechnik; Personalauswahl; Personalentscheidung; Faktorenanalyse
Abstract	Student responses to a large number of constructed response items in three Math and three Reading tests were scored on two occasions using three ways of assigning raters: single reader scoring, a different reader for each response (item-specific), and three readers each scoring a rater item block (RIB) containing approximately one-third of a student's responses. Multiple group confirmatory factor analyses indicated that the three types of total scores were most frequently tau-equivalent. Factor models fitted on the item responses attributed differences in scores to correlated ratings incurred by the same reader scoring multiple responses. These halo effects contributed to significantly increased single reader mean total scores for three of the tests. The similarity of scores for item-specific and RIB scoring suggests that the effect of rater bias on an examinee's set of responses may be minimized with the use of multiple readers though fewer than the number of items. (Author).
Anmerkungen	Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail: customerservices@blackwellpublishing.com; Web site: http://www.blackwellpublishing.com/jnl_default.asp
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10