A Comparison of Methods for Detecting Examinee Preknowledge of Items

Autor/inn/en	Wang, Xi; Liu, Yang; Robin, Frederic; Guo, Hongwen
Titel	A Comparison of Methods for Detecting Examinee Preknowledge of Items
Quelle	In: International Journal of Testing, 19 (2019) 3, S.207-226 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1530-5058
DOI	10.1080/15305058.2019.1610886
Schlagwörter	Identification; Methods; Test Items; Cheating; Adaptive Testing; Statistical Analysis + Suchen Sie Ihr Suchwort? Identifikation; Identifizierung; Method; Methode; Test content; Testaufgabe; Prellen; Statistische Analyse
Abstract	In an on-demand testing program, some items are repeatedly used across test administrations. This poses a risk to test security. In this study, we considered a scenario wherein a test was divided into two subsets: one consisting of secure items and the other consisting of possibly compromised items. In a simulation study of multistage adaptive testing, we used three methods to detect item preknowledge: a predictive checking method (PCM), a likelihood ratio test (LRT), and an adapted Kullback--Leibler divergence (KLD-A) test. We manipulated four factors: the proportion of compromised items, the stage of adaptive testing at which preknowledge was present, item-parameter estimation error, and the information contained in secure items. The type I error results indicated that the LRT and PCM methods are favored over the KLD-A method because the KLD-A can experience large inflated type I error in many conditions. In regard to power, the LRT and PCM methods displayed a wide range of results, generally from 0.2 to 0.8, depending on the amount of preknowledge and the stage of adaptive testing at which the preknowledge was present. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01