Incorporating Test-Taking Engagement into the Item Selection Algorithm in Low-Stakes Computerized Adaptive Tests

Autor/inn/en	Gorgun, Guher; Bulut, Okan
Titel	Incorporating Test-Taking Engagement into the Item Selection Algorithm in Low-Stakes Computerized Adaptive Tests
Quelle	In: Large-scale Assessments in Education, 11 (2023), Artikel 27 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Gorgun, Guher) ORCID (Bulut, Okan)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
DOI	10.1186/s40536-023-00177-5
Schlagwörter	Computer Assisted Testing; Adaptive Testing; Test Items; Algorithms; Guessing (Tests); Test Wiseness; Accuracy; Item Response Theory; Reaction Time; Selection + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Algorithm; Algorithmus; Erraten; Item-Response-Theorie; Reaktionsvermögen; Auslese
Abstract	In low-stakes assessment settings, students' performance is not only influenced by students' ability level but also their test-taking engagement. In computerized adaptive tests (CATs), disengaged responses (e.g., rapid guesses) that fail to reflect students' true ability levels may lead to the selection of less informative items and thereby contaminate item selection and ability estimation procedures. To date, researchers have developed various approaches to detect and remove disengaged responses after test administration is completed to alleviate the negative impact of low test-taking engagement on test scores. This study proposes an alternative item selection method based on Maximum Fisher Information (MFI) that considers test-taking engagement as a secondary latent trait to select the most optimal items based on both ability and engagement. The results of post-hoc simulation studies indicated that the proposed method could optimize item selection and improve the accuracy of final ability estimates, especially for low-ability students. Overall, the proposed method showed great promise for tailoring CATs based on test-taking engagement. Practitioners are encouraged to consider incorporating engagement into the item selection algorithm to enhance the validity of inferences made from low-stakes CATs. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01