Effects of Marking Method and Rater Experience on ESL Essay Scores and Rater Performance

Autor/in	Barkaoui, Khaled
Titel	Effects of Marking Method and Rater Experience on ESL Essay Scores and Rater Performance
Quelle	In: Assessment in Education: Principles, Policy & Practice, 18 (2011) 3, S.279-293 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0969-594X
Schlagwörter	Writing Evaluation; Writing Tests; Essay Tests; Interrater Reliability; Scoring; Scoring Formulas; Evaluation Methods; Transfer of Training; English (Second Language); Experienced Teachers; Beginning Teachers; Writing Ability; Comparative Analysis; Adult Education + Suchen Sie Ihr Suchwort? Writing test; Schreibtest; Schriftlicher Sprachgebrauch; Interrater-Reliabilität; Bewertung; Scoring rubrics; Auswertungsbogen; Training; Transfer; Ausbildung; English as second language; English; Second Language; Englisch als Zweitsprache; Junior teacher; Junglehrer; Schreibkompetenz; Adult; Adults; Education; Adult basic education; Adult training; Erwachsenenbildung
Abstract	This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers' writing ability estimates, inter-rater agreement, and rater severity and self-consistency across marking methods (holistic vs. analytic) and rater groups (novice vs. experienced). Raters tended to be less severe with analytic marking. Holistic marking led to a higher level of inter-rater agreement, while analytic marking led to higher self-consistency. The two rater groups differed in terms of severity, inter-rater agreement and self-consistency. Furthermore, the two marking methods performed differently across the two rater groups. Overall, the findings suggest that the two marking methods might be useful for different assessment purposes, contexts and raters. Implications for ESL writing assessment practice and research are discussed. (Contains 5 notes and 3 tables.) (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10