Linking Assessment to Real Life Practice -- Comparing Work Based Assessments and Objective Structured Clinical Examinations Using Mystery Shopping

Autor/inn/en	Angelina Lim; Sunanthiny Krishnan; Harjit Singh; Simon Furletti; Mahbub Sarkar; Derek Stewart; Daniel Malone
Titel	Linking Assessment to Real Life Practice -- Comparing Work Based Assessments and Objective Structured Clinical Examinations Using Mystery Shopping
Quelle	In: Advances in Health Sciences Education, 29 (2024) 3, S. 859-878Infoseite zur Zeitschrift PDF als Volltext Link als defekt meldenVerfügbarkeit
Zusatzinformation	ORCID (Angelina Lim)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-023-10284-1
Schlagwörter	Forschungsbericht; Allied Health Occupations Education; Clinical Experience; Performance Based Assessment; Vocational Evaluation; Pharmaceutical Education; Undergraduate Students; Objective Tests; Occupational Tests; Situational Tests + Suchen Sie Ihr Suchwort? Leistungsermittlung; Objektiver Test; Berufseignungsprüfung; Situationsansatz
Abstract	Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Begutachtung	Peer reviewed
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2025/2/06