A Summary of OCR's Pilots of the Use of Comparative Judgement in Setting Grade Boundaries

Autor/inn/en	Benton, Tom; Gill, Tim; Hughes, Sarah; Leech, Tony
Titel	A Summary of OCR's Pilots of the Use of Comparative Judgement in Setting Grade Boundaries
Quelle	In: Research Matters, (2022) 33, S.10-30 (21 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1755-6031
Schlagwörter	Comparative Analysis; Grades (Scholastic); Academic Achievement; Grading; Student Evaluation + Suchen Sie Ihr Suchwort? Notenspiegel; Schulleistung; Notengebung; Schulnote; Studentische Bewertung
Abstract	The rationale for the use of comparative judgement (CJ) to help set grade boundaries is to provide a way of using expert judgement to identify and uphold certain minimum standards of performance rather than relying purely on statistical approaches such as comparable outcomes. This article summarises the results of recent trials of using CJ for this purpose in terms of how much difference it might have made to the positions of grade boundaries, the reported precision of estimates and the amount of time that was required from expert judges. The results show that estimated grade boundaries from a CJ approach tend to be fairly close to those that were set (using other forms of evidence) in practice. However, occasionally, CJ results displayed small but significant differences with existing boundary locations. This implies that adopting a CJ approach to awarding would have a noticeable impact on awarding decisions but not such a large one as to be implausible. This article also demonstrates that implementing CJ using simplified methods (described by Benton, Cunningham et al, 2020) achieves the same precision as alternative CJ approaches, but in less time. On average, each CJ exercise required roughly 30 judge-hours across all judges. (As Provided).
Anmerkungen	Cambridge University Press & Assessment. Shaftesbury Road Cambridge CB2 8EA. Tel: 44-1223-553311; e-mail: directcs@cambridge.org; Web site: https://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01