Assessment of Factual Recall and Higher-Order Cognitive Domains in an Open-Book Medical School Examination

Autor/inn/en	Davies, D. J.; McLean, P. F.; Kemp, P. R.; Liddle, A. D.; Morrell, M. J.; Halse, O.; Martin, N. M.; Sam, A. H.
Titel	Assessment of Factual Recall and Higher-Order Cognitive Domains in an Open-Book Medical School Examination
Quelle	In: Advances in Health Sciences Education, 27 (2022) 1, S.147-165 (19 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Davies, D. J.) ORCID (Sam, A. H.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-021-10076-5
Schlagwörter	Medical Education; Tests; Internet; Access to Information; Performance; Recall (Psychology); Knowledge Level; Medical Students + Suchen Sie Ihr Suchwort? Medizinische Ausbildung; Examination; Prüfung; Examen; Achievement; Leistung; Abberufung; Wissensbasis
Abstract	Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom's taxonomy ('Remember', versus 'Understand/Apply'). We predicted higher performance on the OBE, driven by higher performance on 'Remember' questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms 'Open-Book, 'Bloom Category' and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14-2.39), and higher on 'Remember' than 'Understand/Apply' questions (OR 1.13, 95% CI: 1.09-1.19). The difference in performance between 'Remember' and 'Understand/Apply' questions was greater in the OBE than the CBE ('Open-Book' * 'Bloom Category' interaction: OR 1.2, 95% CI: 1.19-1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01