Using Natural Language Processing to Predict Item Response Times and Improve Test Construction

Autor/inn/en	Baldwin, Peter; Yaneva, Victoria; Mee, Janet; Clauser, Brian E.; Ha, Le An
Titel	Using Natural Language Processing to Predict Item Response Times and Improve Test Construction
Quelle	In: Journal of Educational Measurement, 58 (2021) 1, S.4-30 (27 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0655
DOI	10.1111/jedm.12264
Schlagwörter	Natural Language Processing; Prediction; Item Response Theory; Reaction Time; Test Construction; Test Items + Suchen Sie Ihr Suchwort? Natürliche Sprache; Vorhersage; Item-Response-Theorie; Reaktionsvermögen; Testaufbau; Test content; Testaufgabe
Abstract	In this article, it is shown how item text can be represented by (a) 113 features quantifying the text's linguistic characteristics, (b) 16 measures of the extent to which an information-retrieval-based automatic question-answering system finds an item challenging, and (c) through dense word representations (word embeddings). Using a random forests algorithm, these data then are used to train a prediction model for item response times and predicted response times then are used to assemble test forms. Using empirical data from the United States Medical Licensing Examination, we show that timing demands are more consistent across these specially assembled forms than across forms comprising randomly-selected items. Because an exam's timing conditions affect examinee performance, this result has implications for exam fairness whenever examinees are compared with each other or against a common standard. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01