Detecting AI-Generated Essays: The ChatGPT Challenge

Autor/in	Cingillioglu, Ilker
Titel	Detecting AI-Generated Essays: The ChatGPT Challenge
Quelle	In: International Journal of Information and Learning Technology, 40 (2023) 3, S.259-268 (10 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Cingillioglu, Ilker)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2056-4880
DOI	10.1108/IJILT-03-2023-0043
Schlagwörter	Artificial Intelligence; Essays; Integrity; Cheating; Ethics; Identification; Man Machine Systems; Algorithms; Accuracy; Models; Language; Writing (Composition) + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Essay; Aufsatzunterricht; Integrität; Prellen; Ethik; Identifikation; Identifizierung; Mensch-Maschine-System; Algorithm; Algorithmus; Analogiemodell; Languages; Sprache; Schreibübung
Abstract	Purpose: With the advent of ChatGPT, a sophisticated generative artificial intelligence (AI) tool, maintaining academic integrity in all educational settings has recently become a challenge for educators. This paper discusses a method and necessary strategies to confront this challenge. Design/methodology/approach: In this study, a language model was defined to achieve high accuracy in distinguishing ChatGPT-generated essays from human written essays with a particular focus on "not falsely" classifying genuinely human-written essays as AI-generated (Negative). Findings: Via support vector machine (SVM) algorithm 100% accuracy was recorded for identifying human generated essays. The author discussed the key use of Recall and F2 score for measuring classification performance and the importance of eliminating False Negatives and making sure that no actual human generated essays are incorrectly classified as AI generated. The results of the proposed model's classification algorithms were compared to those of AI-generated text detection software developed by OpenAI, GPTZero and Copyleaks. Practical implications: AI-generated essays submitted by students can be detected by teachers and educational designers using the proposed language model and machine learning (ML) classifier at a high accuracy. Human (student)-generated essays can and must be correctly identified with 100% accuracy even if the overall classification accuracy performance is slightly reduced. Originality/value: This is the first and only study that used an n-gram bag-of-words (BOWs) discrepancy language model as input for a classifier to make such prediction and compared the classification results of other AI-generated text detection software in an empirical way. (As Provided).
Anmerkungen	Emerald Publishing Limited. Howard House, Wagon Lane, Bingley, West Yorkshire, BD16 1WA, UK. Tel: +44-1274-777700; Fax: +44-1274-785201; e-mail: emerald@emeraldinsight.com; Web site: http://www.emerald.com/insight
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01