Opening the Black Box of Selection

Autor/inn/en	Schreurs, Sanne; Cleutjens, Kitty; Collares, Carlos F.; Cleland, Jennifer; oude Egbrink, Mirjam G. A.
Titel	Opening the Black Box of Selection
Quelle	In: Advances in Health Sciences Education, 25 (2020) 2, S.363-382 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Schreurs, Sanne) ORCID (Cleutjens, Kitty) ORCID (Collares, Carlos F.) ORCID (Cleland, Jennifer) ORCID (oude Egbrink, Mirjam G. A.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-019-09925-1
Schlagwörter	Medical Schools; Selection Criteria; College Admission; Prediction; Guidelines; Construct Validity; Video Technology; Aptitude Tests; Decision Making; Test Items; Writing Tests; College Applicants + Suchen Sie Ihr Suchwort? Medizinische Ausbildung; Selection criterion; Auslesekriterium; Hochschulzugang; Hochschulzulassung; Zulassung; Vorhersage; Richtlinien; Aptitude test; Eignungsprüfung; Eignungstest; Decision-making; Entscheidungsfindung; Test content; Testaufgabe; Writing test; Schreibtest; College applications; Studienbewerber
Abstract	Medical school selection is currently in the paradoxical situation in which selection tools may predict study outcomes, but which constructs are actually doing the predicting is unknown (the 'black box of selection'). Therefore, our research focused on those constructs, answering the question: do the internal structures of the tests in an outcome-based selection procedure reflect the content that was intended to be measured? Downing's validity framework was applied to organize evidence for construct validity, focusing on evidence related to content and internal structure. The applied selection procedure was a multi-tool, CanMEDS-based procedure comprised of a video-based situational judgement test (focused on (inter)personal competencies), and a written aptitude test (reflecting a broader array of CanMEDS competencies). First, we examined content-related evidence pertaining to the creation and application of the competency-based selection blueprint and found that the set-up of the selection procedure was a robust, transparent and replicable process. Second, the internal structure of the selection tests was investigated by connecting applicants' performance on the selection tests to the predetermined blueprint using cognitive diagnostic modeling. The data indicate 89% overlap between the expected and measured constructs. Our results support the notion that the focus placed on creating the right content and following a competency-blueprint was effective in terms of internal structure: most items measured what they were intended to measure. This way of linking a predetermined blueprint to the applicants' results sheds light into the 'black box of selection' and can be used to support the construct validity of selection procedures. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01