Can Machine Scoring Deal with Broad and Open Writing Tests as Well as Human Readers?

Autor/in	McCurry, Doug
Titel	Can Machine Scoring Deal with Broad and Open Writing Tests as Well as Human Readers?
Quelle	In: Assessing Writing, 15 (2010) 2, S.118-129 (12 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1075-2935
DOI	10.1016/j.asw.2010.04.002
Schlagwörter	Writing Tests; Scoring; Interrater Reliability; Computer Assisted Testing; Essay Tests; Computer Software; Computer Software Evaluation; Writing Evaluation; Writing Instruction; Test Scoring Machines; National Assessment of Educational Progress + Suchen Sie Ihr Suchwort? Writing test; Schreibtest; Bewertung; Interrater-Reliabilität; Schriftlicher Sprachgebrauch; Softwareanalyse; Schreibunterricht
Abstract	This article considers the claim that machine scoring of writing test responses agrees with human readers as much as humans agree with other humans. These claims about the reliability of machine scoring of writing are usually based on specific and constrained writing tasks, and there is reason for asking whether machine scoring of writing requires specific and constrained tasks to produce results that mimic human judgements. The conclusion of a National Assessment of Educational Progress (NAEP) report on the online assessment of writing that "the automated scoring of essay responses did not agree with the scores awarded by human readers" is discussed. The article presents the results of a trial in which two software programmes for scoring writing test responses were compared with the results of the human scoring of a broad and open writing test. The trial showed that "automated essay scoring" (AES) did not grade the broad and open writing task responses as reliably as human markers. (Contains 6 tables.) (As Provided).
Anmerkungen	Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10