Effective Evaluation of Online Learning Interventions with Surrogate Measures [Konferenzbericht] Paper presented at the International Conference on Educational Data Mining (EDM) (16th, Bengaluru, India, Jul 11-14, 2023).

Autor/inn/en	Prihar, Ethan; Vanacore, Kirk; Sales, Adam; Heffernan, Neil
Titel	Effective Evaluation of Online Learning Interventions with Surrogate Measures [Konferenzbericht] Paper presented at the International Conference on Educational Data Mining (EDM) (16th, Bengaluru, India, Jul 11-14, 2023).
Quelle	(2023), (7 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Electronic Learning; Intervention; Instructional Effectiveness; Data Collection; Models; Measurement + Suchen Sie Ihr Suchwort? Unterrichtserfolg; Data capture; Datensammlung; Analogiemodell; Messverfahren
Abstract	There is a growing need to empirically evaluate the quality of online instructional interventions at scale. In response, some online learning platforms have begun to implement rapid A/B testing of instructional interventions. In these scenarios, students participate in series of randomized experiments that evaluate problem-level interventions in quick succession, which makes it difficult to discern the effect of any particular intervention on their learning. Therefore, distal measures of learning such as posttests may not provide a clear understanding of which interventions are effective, which can lead to slow adoption of new instructional methods. To help discern the effectiveness of instructional interventions, this work uses data from 26,060 clickstream sequences of students across 31 different online educational experiments exploring 51 different research questions and the students' posttest scores to create and analyze different proximal surrogate measures of learning that can be used at the problem level. Through feature engineering and deep learning approaches, next-problem correctness was determined to be the best surrogate measure. As more data from online educational experiments are collected, model based surrogate measures can be improved, but for now, next-problem correctness is an empirically effective proximal surrogate measure of learning for analyzing rapid problem-level experiments. [For the complete proceedings, see ED630829. Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN).] (As Provided).
Anmerkungen	International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01