Comparing the Performance of Five Multidimensional CAT Selection Procedures with Different Stopping Rules

Autor/in	Yao, Lihua
Titel	Comparing the Performance of Five Multidimensional CAT Selection Procedures with Different Stopping Rules
Quelle	In: Applied Psychological Measurement, 37 (2013) 1, S.3-23 (21 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0146-6216
DOI	10.1177/0146621612455687
Schlagwörter	Computer Assisted Testing; Adaptive Testing; Test Items; Selection; Test Length; Item Response Theory; Scores; Test Reliability; Error of Measurement; Simulation; Armed Forces Qualification Test + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Auslese; Item-Response-Theorie; Testreliabilität; Messfehler; Simulation program; Simulationsprogramm
Abstract	Through simulated data, five multidimensional computerized adaptive testing (MCAT) selection procedures with varying test lengths are examined and compared using different stopping rules. Fixed item exposure rates are used for all the items, and the Priority Index (PI) method is used for the content constraints. Two stopping rules, standard error (SE) and predicted standard error reduction (PSER), are proposed; each MCAT selection process is stopped if either the required precision has been achieved or the selected number of items has reached the maximum limit. The five procedures are as follows: minimum angle (Ag), volume (Vm), minimize the error variance of the linear combination (V[subscript 1]), minimize the error variance of the composite score with the optimized weight (V[subscript 2]), and Kullback-Leibler (KL) information. The recovery for the domain scores or content scores and their overall score, test length, and test reliability are compared across the five MCAT procedures and between the two stopping rules. It is found that the two stopping rules are implemented successfully and that KL uses the least number of items to reach the same precision level, followed by Vm; Ag uses the largest number of items. On average, to reach a precision of SE = 0.35, 40, 55, 63, 63, and 82 items are needed for KL, Vm, V[subscript 1], V[subscript 2], and Ag, respectively, for the "S"E stopping rule. PSER yields 38, 45, 53, 58, and 68 items for KL, Vm, V[subscript 1], V[subscript 2], and Ag, respectively; PSER yields only slightly worse results than "SE", but with much fewer items. Overall, KL is recommended for varying-length MCAT. (Contains 2 tables and 4 figures.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10