Applying Cognitive Theory to the Human Essay Rating Process

Autor/inn/en	Finn, Bridgid; Arslan, Burcu; Walsh, Matthew
Titel	Applying Cognitive Theory to the Human Essay Rating Process
Quelle	In: Applied Measurement in Education, 33 (2020) 3, S.223-233 (11 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0895-7347
DOI	10.1080/08957347.2020.1750405
Schlagwörter	Epistemology; Essay Tests; Evaluators; Cognitive Processes; Scoring; Accuracy; English (Second Language); Language Tests; Second Language Learning; Graduate Study; College Entrance Examinations; Predictor Variables; Skill Development; Test of English as a Foreign Language; Graduate Record Examinations + Suchen Sie Ihr Suchwort? Erkenntnistheorie; Schriftlicher Sprachgebrauch; Cognitive process; Kognitiver Prozess; Bewertung; English as second language; English; Second Language; Englisch als Zweitsprache; Language test; Sprachtest; Zweitsprachenerwerb; Aufbaustudium; Graduiertenstudium; Hauptstudium; Aufnahmeprüfung; Prädiktor; Kompetenzentwicklung; Qualifikationsentwicklung
Abstract	To score an essay response, raters draw on previously trained skills and knowledge about the underlying rubric and score criterion. Cognitive processes such as remembering, forgetting, and skill decay likely influence rater performance. To investigate how forgetting influences scoring, we evaluated raters' scoring accuracy on TOEFL and GRE essays. We used binomial linear mixed effect models to evaluate how the effect of various predictors such as time spent scoring each response and days between scoring sessions relate to scoring accuracy. Results suggest that for both nonoperational (i.e., calibration samples completed prior to a scoring session) and operational scoring (i.e., validity samples interspersed among actual student responses), the number of days in a scoring gap negatively affects performance. The findings, as well as other results from the models are discussed in the context of cognitive influences on knowledge and skill retention. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01