Performance of a Generic Approach in Automated Essay Scoring

Autor/inn/en	Attali, Yigal; Bridgeman, Brent; Trapani, Catherine
Titel	Performance of a Generic Approach in Automated Essay Scoring
Quelle	In: Journal of Technology, Learning, and Assessment, 10 (2010) 3, (17 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1540-2525
Schlagwörter	Writing Evaluation; Writing Tests; Scoring; Test Scoring Machines; Comparative Testing; Computer Assisted Testing; Weighted Scores; Equated Scores; Essays; Standardized Tests; Multiple Regression Analysis; Pattern Recognition; Item Analysis; Evaluation Methods; Graduate Record Examinations; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Writing test; Schreibtest; Bewertung; Essay; Aufsatzunterricht; Standadised tests; Standardisierter Test; Mustererkennung; Itemanalyse
Abstract	A generic approach in automated essay scoring produces scores that have the same meaning across all prompts, existing or new, of a writing assessment. This is accomplished by using a single set of linguistic indicators (or features), a consistent way of combining and weighting these features into essay scores, and a focus on features that are not based on prompt-specific information or vocabulary. This approach has both logistical and validity-related advantages. This paper evaluates the performance of generic scores in the context of the e-rater[R] automated essay scoring system. Generic scores were compared with prompt-specific scores and scores that included prompt-specific vocabulary features. These comparisons were performed with large samples of essays written to three writing assessments: The GRE General Test argument and issue tasks and the TOEFL independent task. Criteria for evaluation included level of agreement with human scores, discrepancy from human scores across prompts, and correlations with other available scores. Results showed small differences between generic and prompt-specific scores and adequate performance of both types of scores compared to human performance. (Contains 3 tables.) (As Provided).
Anmerkungen	Technology and Assessment Study Collaborative. 332 Campion Hall, Boston College, Chestnut Hill, MA 02467. Tel: 617-552-4521; Fax: 617-552-8419; e-mail: jtal_editor@bc.edu; Web site: http://escholarship.bc.edu/jtla
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10