Harnessing the Potential of Trace Data and Linguistic Analysis to Predict Learner Performance in a Multi-Text Writing Task

Autor/inn/en	Rakovic, Mladen; Iqbal, Sehrish; Li, Tongguang; Fan, Yizhou; Singh, Shaveen; Surendrannair, Surya; Kilgour, Jonathan; Graaf, Joep; Lim, Lyn; Molenaar, Inge; Bannert, Maria; Moore, Johanna; Gaševic, Dragan
Titel	Harnessing the Potential of Trace Data and Linguistic Analysis to Predict Learner Performance in a Multi-Text Writing Task
Quelle	In: Journal of Computer Assisted Learning, 39 (2023) 3, S.703-718 (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Rakovic, Mladen) ORCID (Fan, Yizhou)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0266-4909
DOI	10.1111/jcal.12769
Schlagwörter	Artificial Intelligence; Essays; High Achievement; Writing Achievement; Identification; Graduate Students; Writing (Composition); Natural Language Processing; Writing Evaluation + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Essay; Aufsatzunterricht; Identifikation; Identifizierung; Graduate Study; Student; Students; Aufbaustudium; Graduiertenstudium; Hauptstudium; Studentin; Schreibübung; Natürliche Sprache
Abstract	Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low- and high-performing essays in a multi-text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi-text writing assignment. We utilised learners' trace data to obtain process features and state-of-the-art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re-reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi-text writing task to help them do well in the task. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01