An Evaluation of Item Response Theory Classification Accuracy and Consistency Indices

Autor/inn/en	Wyse, Adam E.; Hao, Shiqi
Titel	An Evaluation of Item Response Theory Classification Accuracy and Consistency Indices
Quelle	In: Applied Psychological Measurement, 36 (2012) 7, S.602-624 (23 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0146-6216
DOI	10.1177/0146621612451522
Schlagwörter	Item Response Theory; Classification; Accuracy; Reliability; Computation; Maximum Likelihood Statistics; Cutting Scores; Test Length; High School Students; Achievement Tests; Reading Tests; Mathematics Tests; Michigan + Suchen Sie Ihr Suchwort? Item-Response-Theorie; Classification system; Klassifikation; Klassifikationssystem; Reliabilität; High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Lesetest
Abstract	This article introduces two new classification consistency indices that can be used when item response theory (IRT) models have been applied. The new indices are shown to be related to Rudner's classification accuracy index and Guo's classification accuracy index. The Rudner- and Guo-based classification accuracy and consistency indices are evaluated and compared with estimates from the more commonly applied IRT-recursive procedure using a simulation study and data from two large-scale assessments. Results from the simulation study and practical examples suggested that the Guo- and Rudner-based indices tended to produce estimates that were closer to the simulated values and exceeded those from the IRT-recursive-based procedure. However, results did suggest that the Rudner- and Guo-based indices can have some undesirable features that are important to keep in mind when applying them in practice. The values of the classification accuracy and consistency indices appeared to be affected by a number of factors including the distribution of examinees, test length, the placement of the cut-scores, and the proficiency estimators applied to estimate examinee ability. Suggestions are made that an important part of investigations evaluating classification accuracy and consistency indices should be the creation of figures that show the value of the classification accuracy and classification consistency for individual examinees across the range of possible scores as these figures can help provide indications into subtle and important differences between indices. (Contains 5 tables and 3 figures.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10