Literaturnachweis - Detailanzeige
Autor/inn/en | Abu-Ghazalah, Rashid M.; Dubins, David N.; Poon, Gregory M. K. |
---|---|
Titel | Dissecting Knowledge, Guessing, and Blunder in Multiple Choice Assessments |
Quelle | In: Applied Measurement in Education, 36 (2023) 1, S.80-98 (19 Seiten)Infoseite zur Zeitschrift
PDF als Volltext |
Zusatzinformation | ORCID (Poon, Gregory M. K.) |
Sprache | englisch |
Dokumenttyp | gedruckt; online; Zeitschriftenaufsatz |
ISSN | 0895-7347 |
DOI | 10.1080/08957347.2023.2172017 |
Schlagwörter | Guessing (Tests); Multiple Choice Tests; Probability; Models; Error Patterns; Responses; Item Analysis; Test Items; Undergraduate Students; Biotechnology; Bayesian Statistics; Scores; Self Esteem; Knowledge Level; Test Construction; Foreign Countries; Canada Erraten; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Analogiemodell; Fehlertyp; Itemanalyse; Test content; Testaufgabe; Biotechnologie; Self-esteem; Selbstaufmerksamkeit; Wissensbasis; Testaufbau; Ausland; Kanada |
Abstract | Multiple choice results are inherently probabilistic outcomes, as correct responses reflect a combination of knowledge and guessing, while incorrect responses additionally reflect blunder, a confidently committed mistake. To objectively resolve knowledge from responses in an MC test structure, we evaluated probabilistic models that explicitly account for guessing, knowledge, and blunder using eight assessments (>9,000 responses) from an undergraduate biotechnology curriculum. A Bayesian implementation of the models, aimed at assessing their robustness to prior beliefs in examinee knowledge, showed that explicit estimators of knowledge are markedly sensitive to prior beliefs with scores as sole input. To overcome this limitation, we examined self-ranked confidence as a proxy knowledge indicator. For our test set, three levels of confidence resolved test performance. Responses rated as least confident were correct more frequently than expected from random selection, reflecting partial knowledge, but were balanced by blunder among the most confident responses. By translating evidence-based guessing and blunder rates to pass marks that statistically qualify a desired level of examinee knowledge, our approach finds practical utility in test analysis and design. (As Provided). |
Anmerkungen | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
Update | 2024/1/01 |