A Systematic Review of Automated Writing Evaluation Systems

Autor/inn/en	Huawei, Shi; Aryadoust, Vahid
Titel	A Systematic Review of Automated Writing Evaluation Systems
Quelle	In: Education and Information Technologies, 28 (2023) 1, S.771-795 (25 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Aryadoust, Vahid)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1360-2357
DOI	10.1007/s10639-022-11200-7
Schlagwörter	Writing Evaluation; Writing Tests; Computer Assisted Testing; Automation; Scoring; Educational Trends; Test Validity; Natural Language Processing + Suchen Sie Ihr Suchwort? Writing test; Schreibtest; Bewertung; Bildungsentwicklung; Testvalidität; Natürliche Sprache
Abstract	Automated writing evaluation (AWE) systems are developed based on interdisciplinary research and technological advances such as natural language processing, computer sciences, and latent semantic analysis. Despite a steady increase in research publications in this area, the results of AWE investigations are often mixed, and their validity may be questionable. To yield a deeper understanding of the validity of AWE systems, we conducted a systematic review of the empirical AWE research. Using Scopus, we identified 105 published papers on AWE scoring systems and coded them within an argument-based validation framework. The major findings are: (i) AWE scoring research had a rising trend, but was heterogeneous in terms of the language environments, ecological settings, and educational level; (ii) a disproportionate number of studies were carried out on each validity inference, with the evaluation inference receiving the most research attention, and the domain description inference being the neglected one, and (iii) most studies adopted quantitative methods and yielded positive results that backed each inference, while some studies also presented counterevidence. Lack of research on the domain description (i.e., the correspondence between the AWE systems and real-life writing tasks) combined with the heterogeneous contexts indicated that construct representation in the AWE scoring field needs extensive investigation. Implications and directions for future research are also discussed. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01