Peer Overmarking and Insufficient Diagnosticity: The Impact of the Rating Method for Peer Assessment

Autor/inn/en	Van Meenen, Florence; Coertjens, Liesje; Van Nes, Marie-Claire; Verschuren, Franck
Titel	Peer Overmarking and Insufficient Diagnosticity: The Impact of the Rating Method for Peer Assessment
Quelle	In: Advances in Health Sciences Education, 27 (2022) 4, S.1049-1066 (18 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Van Meenen, Florence)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-022-10130-w
Schlagwörter	Evaluation Methods; Peer Evaluation; Accuracy; Evaluation Criteria; Undergraduate Students; Essays; Pollution; Scores; Validity; Reliability + Suchen Sie Ihr Suchwort? Essay; Aufsatzunterricht; Schadstoffbelastung; Gültigkeit; Reliabilität
Abstract	The present study explores two rating methods for peer assessment (analytical rating using criteria and comparative judgement) in light of concurrent validity, reliability and insufficient diagnosticity (i.e. the degree to which substandard work is recognised by the peer raters). During a second-year undergraduate course, students wrote a one-page essay on an air pollutant. A first cohort (N = 260) relied on analytical rating using criteria to assess their peers' essays. A total of 1297 evaluations were made, and each essay received at least four peer ratings. Results indicate a small correlation between peer and teacher marks, and three essays of substandard quality were not recognised by the group of peer raters. A second cohort (N = 230) used comparative judgement. They completed 1289 comparisons, from which a rank order was calculated. Results suggest a large correlation between the university teacher marks and the peer scores and acceptable reliability of the rank order. In addition, the three essays of substandard quality were discerned as such by the group of peer raters. Although replication research is warranted, the results provide the first evidence that, when peer raters overmark and fail to identify substandard work using analytical rating with criteria, university teachers may consider changing the rating method of the peer assessment to comparative judgement. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01