Comparing Large-Scale Assessments in Two Proctoring Modalities with Interactive Log Data Analysis

Autor/inn/en	Shin, Jinnie; Guo, Qi; Morin, Maxim
Titel	Comparing Large-Scale Assessments in Two Proctoring Modalities with Interactive Log Data Analysis
Quelle	In: Educational Measurement: Issues and Practice, 42 (2023) 4, S.66-80 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Shin, Jinnie)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/emip.12582
Schlagwörter	Foreign Countries; High Stakes Tests; Computer Assisted Testing; Licensing Examinations (Professions); Medical Education; Supervision; Reaction Time; Canada + Suchen Sie Ihr Suchwort? Ausland; Medizinische Ausbildung; Reaktionsvermögen; Kanada
Abstract	With the increased restrictions on physical distancing due to the COVID-19 pandemic, remote proctoring has emerged as an alternative to traditional onsite proctoring to ensure the continuity of essential assessments, such as computer-based medical licensing exams. Recent literature has highlighted the significant impact of different proctoring modalities on examinees' test experience, including factors like response-time data. However, the potential influence of these differences on test performance has remained unclear. One limitation in the current literature is the lack of a rigorous learning analytics framework to evaluate the comparability of computer-based exams delivered using various proctoring settings. To address this gap, the current study aims to introduce a machine-learning-based framework that analyzes computer-generated response-time data to investigate the association between proctoring modalities in high-stakes assessments. We demonstrated the effectiveness of this framework using empirical data collected from a large-scale high-stakes medical licensing exam conducted in Canada. By applying the machine-learning-based framework, we were able to extract examinee-specific response-time data for each proctoring modality and identify distinct time-use patterns among examinees based on their proctoring modality. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01