A Monte Carlo Simulation Investigating the Validity and Reliability of Ability Estimation in Item Response Theory with Speeded Computer Adaptive Tests

Autor/inn/en	Schmitt, T. A.; Sass, D. A.; Sullivan, J. R.; Walker, C. M.
Titel	A Monte Carlo Simulation Investigating the Validity and Reliability of Ability Estimation in Item Response Theory with Speeded Computer Adaptive Tests
Quelle	In: International Journal of Testing, 10 (2010) 3, S.230-261 (32 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1530-5058
Schlagwörter	Monte Carlo Methods; Simulation; Computer Assisted Testing; Adaptive Testing; Timed Tests; Item Response Theory; Validity; Reliability; Computation; Bias; Error of Measurement + Suchen Sie Ihr Suchwort? Monte-Carlo-Methode; Simulation program; Simulationsprogramm; Item-Response-Theorie; Gültigkeit; Reliabilität; Messfehler
Abstract	Imposed time limits on computer adaptive tests (CATs) can result in examinees having difficulty completing all items, thus compromising the validity and reliability of ability estimates. In this study, the effects of speededness were explored in a simulated CAT environment by varying examinee response patterns to end-of-test items. Expectedly, ability estimates became increasingly negatively biased as the CAT became more speeded, with the bias magnitude depending on the speededness condition. Ability estimates for higher-ability examinees were also influenced more than estimates for low- and middle-ability examinees with the realistic item pool. This finding is likely linked to test operational characteristics since these results were not replicated with an ideal item pool. In any case, the CAT is relatively robust to the speededness assumption, assuming that only a few items are speeded. Results also suggested that if test developers score the test at the point of speededness, ability estimates remain unbiased. This study's findings help inform test developers and test takers about the effects of test speededness within a CAT environment. Additional implications for educators and researchers are discussed. (Contains 2 tables, 3 figures and 2 notes.) (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10