Beyond Item Analysis: Connecting Student Behaviour and Performance Using E-Assessment Logs

Autor/inn/en	Lahza, Hatim; Smith, Tammy G.; Khosravi, Hassan
Titel	Beyond Item Analysis: Connecting Student Behaviour and Performance Using E-Assessment Logs
Quelle	In: British Journal of Educational Technology, 54 (2023) 1, S.335-354 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Lahza, Hatim) ORCID (Khosravi, Hassan)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0007-1013
DOI	10.1111/bjet.13270
Schlagwörter	Medical Students; Evaluation; Computer Assisted Testing; Time Factors (Learning); Test Items; Difficulty Level; Student Behavior + Suchen Sie Ihr Suchwort? Evaluierung; Test content; Testaufgabe; Schwierigkeitsgrad; Student behaviour; Schülerverhalten
Abstract	Traditional item analyses such as classical test theory (CTT) use exam-taker responses to assessment items to approximate their difficulty and discrimination. The increased adoption by educational institutions of electronic assessment platforms (EAPs) provides new avenues for assessment analytics by capturing detailed logs of an exam-taker's journey through their exam. This paper explores how logs created by EAPs can be employed alongside exam-taker responses and CTT to gain deeper insights into exam items. In particular, we propose an approach for deriving features from exam logs for approximating item difficulty and discrimination based on exam-taker behaviour during an exam. Items for which difficulty and discrimination differ significantly between CTT analysis and our approach are flagged through outlier detection for independent academic review. We demonstrate our approach by analysing de-identified exam logs and responses to assessment items of 463 medical students enrolled in a first-year biomedical sciences course. The analysis shows that the number of times an exam-taker visits an item before selecting a final response is a strong indicator of an item's difficulty and discrimination. Scrutiny by the course instructor of the seven items identified as outliers suggests our log-based analysis can provide insights beyond what is captured by traditional item analyses. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01