A Human-Centric Automated Essay Scoring and Feedback System for the Development of Ethical Reasoning

Autor/inn/en	Lee, Alwyn Vwen Yen; Luco, Andrés Carlos; Tan, Seng Chee
Titel	A Human-Centric Automated Essay Scoring and Feedback System for the Development of Ethical Reasoning
Quelle	In: Educational Technology & Society, 26 (2023) 1, S.147-159 (13 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1176-3647
Schlagwörter	Essays; Scoring; Writing Evaluation; Computer Software; Feedback (Response); Writing Instruction; Artificial Intelligence; Humanism; Outcomes of Education; Ethics; Moral Values; Persuasive Discourse; Online Courses; Large Group Instruction; Time Management; Undergraduate Students; College Faculty; Algorithms; Interrater Reliability; Accuracy; Peer Evaluation; Scoring Rubrics; Formative Evaluation; Logical Thinking; Knowledge Economy + Suchen Sie Ihr Suchwort? Essay; Aufsatzunterricht; Bewertung; Schreibunterricht; Künstliche Intelligenz; Humanismus; Lernleistung; Schulerfolg; Ethik; Moral value; Ethischer Wert; Persuasion; Persuasive Kommunikation; Online course; Online-Kurs; Zeitmanagement; Fakultät; Algorithm; Algorithmus; Interrater-Reliabilität; Scoring formulas; Auswertungsbogen; Knowledge society; Economy; Wissensgesellschaft; Wirtschaft
Abstract	Although artificial Intelligence (AI) is prevalent and impacts facets of daily life, there is limited research on responsible and humanistic design, implementation, and evaluation of AI, especially in the field of education. Afterall, learning is inherently a social endeavor involving human interactions, rendering the need for AI designs to be approached from a humanistic perspective, or human-centered AI (HAI). This study focuses on the use of essays as a principal means for assessing learning outcomes, through students' writing in subjects that require arguments and justifications, such as ethics and moral reasoning. We considered AI with a human and student-centric design for formative assessment, using an automated essay scoring (AES) and feedback system to address issues of running an online course with large enrolment and to provide efficient feedback to students with substantial time savings for the instructor. The development of the AES system occurred over four phases as part of an iterative design cycle. A mixed-method approach was used, allowing instructors to qualitatively code subsets of data for training a machine learning model based on the Random Forest algorithm. This model was subsequently used to automatically score more essays at scale. Findings show substantial agreement on inter-rater reliability before model training was conducted with acceptable training accuracy. The AES system's performance was slightly less accurate than human raters but is improvable over multiple iterations of the iterative design cycle. This system has allowed instructors to provide formative feedback, which was not possible in previous runs of the course. (As Provided).
Anmerkungen	International Forum of Educational Technology & Society. Available from: National Yunlin University of Science and Technology. No. 123, Section 3, Daxue Road, Douliu City, Yunlin County, Taiwan 64002. e-mail: journal.ets@gmail.com; Web site: https://www.j-ets.net/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01