Conceptualizing Rater Judgments and Rating Processes for Rater-Mediated Assessments

Autor/inn/en	Wang, Jue; Engelhard, George, Jr.
Titel	Conceptualizing Rater Judgments and Rating Processes for Rater-Mediated Assessments
Quelle	In: Journal of Educational Measurement, 56 (2019) 3, S.582-609 (28 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0655
DOI	10.1111/jedm.12226
Schlagwörter	Evaluative Thinking; Models; Measurement; Achievement; Accuracy; Item Response Theory; Achievement Rating + Suchen Sie Ihr Suchwort? Analogiemodell; Messverfahren; Performance; Leistung; Item-Response-Theorie; Achievement; Rating; Beurteilung; Leistungsbeurteilung
Abstract	Rater-mediated assessments exhibit scoring challenges due to the involvement of human raters. The quality of human ratings largely determines the reliability, validity, and fairness of the assessment process. Our research recommends that the evaluation of ratings should be based on two aspects: a theoretical model of human judgment and an appropriate measurement model for evaluating these judgments. In rater-mediated assessments, the underlying constructs and response processes may require the use of different rater judgment models and the application of different measurement models. We describe the use of Brunswik's lens model as an organizing theme for conceptualizing human judgments in rater-mediated assessments. The constructs vary depending on which distal variables are identified in the lens models for the underlying rater-mediated assessment. For example, one lens model can be developed to emphasize the measurement of student proficiency, while another lens model can stress the evaluation of rater accuracy. Next, we describe two measurement models that reflect different response processes (cumulative and unfolding) from raters: Rasch and hyperbolic cosine models. Future directions for the development and evaluation of rater-mediated assessments are suggested. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01