A Short Note on Optimizing Cost-Generalizability via a Machine-Learning Approach

Autor/inn/en	Jiang, Zhehan; Shi, Dexin; Distefano, Christine
Titel	A Short Note on Optimizing Cost-Generalizability via a Machine-Learning Approach
Quelle	In: Educational and Psychological Measurement, 81 (2021) 6, S.1221-1233 (13 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Shi, Dexin) ORCID (Distefano, Christine)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/0013164421992112
Schlagwörter	Artificial Intelligence; Generalizability Theory; Objective Tests; Foreign Countries; Medical Education; Costs; Test Reliability; Medical Students; Mexico (Mexico City) + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Objektiver Test; Ausland; Medizinische Ausbildung; Cost; Kosten; Testreliabilität
Abstract	The costs of an objective structured clinical examination (OSCE) are of concern to health profession educators globally. As OSCEs are usually designed under generalizability theory (G-theory) framework, this article proposes a machine-learning-based approach to optimize the costs, while maintaining the minimum required generalizability coefficient, a reliability-like index in G-theory. The authors adopted G-theory parameters yielded from an OSCE hosted by a medical school, reproduced the generalizability coefficients to prepare for optimizing manipulations, applied simulated annealing algorithm to calculate the number of facet levels minimizing the associated costs, and conducted the analysis in various conditions via computer simulation. With a given generalizability coefficient, the proposed approach, virtually an instrument of decision-making supports, found the optimal solution for the OSCE such that the associated costs were minimized. The computer simulation results showed how the cost reductions varied with different levels of required generalizability coefficients. Machine learning-based approaches can be used in conjunction with psychometric modeling to help planning assessment tasks more scientifically. The proposed approach is easy to adopt into practice and customize in alignment with specific testing designs. While these results are encouraging, the possible pitfalls such as algorithmic convergences' failure and inadequate cost assumptions should also be avoided. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01