Measures of Relative Metacognitive Accuracy Are Confounded with Task Performance in Tasks That Permit Guessing

Autor/inn/en	Vuorre, Matti; Metcalfe, Janet
Titel	Measures of Relative Metacognitive Accuracy Are Confounded with Task Performance in Tasks That Permit Guessing
Quelle	In: Metacognition and Learning, 17 (2022) 2, S.269-291 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Vuorre, Matti) Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1556-1623
DOI	10.1007/s11409-020-09257-1
Schlagwörter	Metacognition; Accuracy; Memory; Multiple Choice Tests; Guessing (Tests); Self Esteem; Decision Making; Self Evaluation (Individuals); Task Analysis; Monte Carlo Methods + Suchen Sie Ihr Suchwort? Meta cognitive ability; Meta-cognition; Metakognitive Fähigkeit; Metakognition; Gedächtnis; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Erraten; Self-esteem; Selbstaufmerksamkeit; Decision-making; Entscheidungsfindung; Aufgabenanalyse; Monte-Carlo-Methode
Abstract	This article investigates the concern that assessment of metacognitive resolution (or relative accuracy--often evaluated by gamma correlations or signal detection theoretic measures such as d[subscript a]) is vulnerable to an artifact due to guessing that differentially impacts low as compared to high performers on tasks that involve multiple-choice testing. Metacognitive resolution refers to people's ability to tell, via confidence judgments, their correct answers apart from incorrect answers, and is theorized to be an important factor in learning. Resolution--the trial-by-trial association between response accuracy and confidence in that response's accuracy--is a distinct ability from knowledge, or accuracy, and instead indicates a higher-order self-evaluation. It is therefore important that measures of resolution are independent of domain-knowledge accuracy. We conducted six experiments that revealed a positive correlation between metacognitive resolution and performance in multiple-choice mathematics testing. Monte Carlo simulations indicated, however, that resolution metrics are increasingly negatively biased with decreasing performance, because multiple-choice tasks permit correct guessing. We, therefore, argue that the observed positive correlations were probably attributable to an artifact rather than a true correlation between psychological abilities. A final experiment supported the guessing-related confound hypothesis: Resolution and performance were positively correlated in multiple-choice testing, but not in free-response testing. This study brings to light a previously underappreciated limitation in assessing metacognitive resolution and its relation to task performance in criterion tasks that may involve guessing. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01