Comparatively Salient: Examining the Influence of Preceding Performances on Assessors' Focus and Interpretations in Written Assessment Comments

Autor/inn/en	Gingerich, Andrea; Schokking, Edward; Yeates, Peter
Titel	Comparatively Salient: Examining the Influence of Preceding Performances on Assessors' Focus and Interpretations in Written Assessment Comments
Quelle	In: Advances in Health Sciences Education, 23 (2018) 5, S.937-959 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Gingerich, Andrea)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1382-4996
DOI	10.1007/s10459-018-9841-2
Schlagwörter	Evaluation Methods; Scores; Intervention; Schemata (Cognition); Competence; Performance; Benchmarking; Evaluators; Cues; Observation; Comparative Analysis + Suchen Sie Ihr Suchwort? Cognition; Schema; Kognition; Kompetenz; Achievement; Leistung; Stichwort; Beobachtung
Abstract	Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise from an effect on the range-frequency of assessors' internal scales or the salience of performance aspects within assessment judgments. As these suggest different potential interventions, we investigated assessors' cognition by using the insight provided by "clusters of consensus" to determine whether any change in the salience of performance aspects was induced by contrast effects. A dataset from a previous experiment contained scores and comments for 3 encounters: 2 with significant contrast effects and 1 without. Clusters of consensus were identified using F-sort and latent partition analysis both when contrast effects were significant and non-significant. The proportion of assessors making similar comments only significantly differed when contrast effects were significant with assessors more frequently commenting on aspects that were dissimilar with the standard of competence demonstrated in the preceding performance. Rather than simply influencing range-frequency of assessors' scales, preceding performances may affect salience of performance aspects through comparative distinctiveness: when juxtaposed with the context some aspects are more distinct and selectively draw attention. Research is needed to determine whether changes in salience indicate biased or improved assessment information. The potential should be explored to augment existing benchmarking procedures in assessor training by cueing assessors' attention through observation of reference performances immediately prior to assessment. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01