ChatGPT in education. An effect in search of a cause.

Autor/inn/en	Weidlich, Joshua; Gasevic, Dragan; Drachsler, Hendrik; Kirschner, Paul
Titel	ChatGPT in education. An effect in search of a cause.
Quelle	In: Journal of computer assisted learning, 41 (2025) 5, 10 S.Infoseite zur Zeitschrift PDF als Volltext kostenfreie Datei (1); PDF als Volltext kostenfreie Datei (2); PDF als Volltext (3) Link als defekt meldenVerfügbarkeit
Sprache	englisch
Dokumenttyp	online; Zeitschriftenaufsatz
ISSN	1365-2729
DOI	10.25656/01:34618 10.1111/jcal.70105
URN	urn:nbn:de:0111-pedocs-346189
Schlagwörter	Artificial Intelligence
Abstract	Background: As researchers rush to investigate the potential of AI tools like ChatGPT to enhance learning, well-documented pitfalls threaten the validity of this emerging research. Issues of media comparison research, where the confounding of instructional methods and technological affordances is unrecognised, may render effects uninterpretable. Objectives: Using a recent meta-analysis by Deng et al. (Computers & Education, 227, 105224) as an example, we revisit key insights from the media/methods debate to highlight recurring conceptual challenges in ChatGPT efficacy studies. Methods: This conceptual article contrasts nascent ChatGPT research with the more established literature on Intelligent Tutoring Systems to identify three non-negotiable considerations for interpretable effects: (1) descriptions of the precise nature of the experimental treatment and (2) the activities of the control group, as well as (3) outcome measures as valid indicators of learning. To provide some initial evidence, we audited a subset of primary experiments included in Deng et al.'s meta-analysis, demonstrating that only a small minority of studies satisfied all three non-negotiable considerations. Results and Conclusions: Loosely defined treatments, mismatched or opaque controls, and outcome measures with unclear links to durable learning obscure causal claims of this emerging literature. Observed gains cannot, at this time, be confidently attributed to ChatGPT, and meta-analytics effect sizes may over- or understate its benefits. Progress, we argue, will require rigorous designs, transparent reporting, and a critical stance toward "fast science." (DIPF/Orig.).
Erfasst von	DIPF \| Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main
Update	2026/3