Gauging Item Alignment through Online Systems While Controlling for Rater Effects

Autor/inn/en	Anderson, Daniel; Irvin, Shawn; Alonzo, Julie; Tindal, Gerald A.
Titel	Gauging Item Alignment through Online Systems While Controlling for Rater Effects
Quelle	In: Educational Measurement: Issues and Practice, 34 (2015) 1, S.22-33 (12 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/emip.12038
Schlagwörter	Test Items; Alignment (Education); Standards; Online Systems; Mathematics Tests; Formative Evaluation; Interrater Reliability; Item Banks + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Standard; Online; Interrater-Reliabilität
Abstract	The alignment of test items to content standards is critical to the validity of decisions made from standards-based tests. Generally, alignment is determined based on judgments made by a panel of content experts with either ratings averaged or via a consensus reached through discussion. When the pool of items to be reviewed is large, or the content-matter experts are broadly distributed geographically, panel methods present significant challenges. This article illustrates the use of an online methodology for gauging item alignment that does not require that raters convene in person, reduces the overall cost of the study, increases time flexibility, and offers an efficient means for reviewing large item banks. Latent trait methods are applied to the data to control for between-rater severity, evaluate intrarater consistency, and provide item-level diagnostic statistics. Use of this methodology is illustrated with a large pool (1,345) of interim-formative mathematics test items. Implications for the field and limitations of this approach are discussed. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01