Implementation Matters: Generalizing Treatment Effects in Education. EdWorkingPaper No. 23-802

Autor/inn/en	Angrist, Noam; Meager, Rachael
Institution	Annenberg Institute for School Reform at Brown University
Titel	Implementation Matters: Generalizing Treatment Effects in Education. EdWorkingPaper No. 23-802
Quelle	(2023), (57 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Teaching Methods; Low Income; Program Implementation; Intervention; Instructional Effectiveness; Foreign Countries; Basic Skills; Numeracy; Literacy; Developing Nations; Achievement Gains; Influences; Learning; Botswana + Suchen Sie Ihr Suchwort? Teaching method; Lehrmethode; Unterrichtsmethode; Niedriglohn; Unterrichtserfolg; Ausland; Basic skill; Grundfertigkeit; Rechenkompetenz; Alphabetisierung; Schreib- und Lesefähigkeit; Developing country; Developing countries; Entwicklungsland; Achievement gain; Leistungssteigerung; Influence; Einfluss; Einflussfaktor; Lernen
Abstract	Targeted instruction is one of the most effective educational interventions in low- and middle-income countries, yet reported impacts vary by an order of magnitude. We study this variation by aggregating evidence from prior randomized trials across five contexts, and use the results to inform a new randomized trial. We find two factors explain most of the heterogeneity in effects across contexts: the degree of implementation (intention-to-treat or treatment-on-the-treated) and program delivery model (teachers or volunteers). Accounting for these implementation factors yields high generalizability, with similar effect sizes across studies. Thus, reporting treatment-on-the-treated effects, a practice which remains limited, can enhance external validity. We also introduce a new Bayesian framework to formally incorporate implementation metrics into evidence aggregation. Results show targeted instruction delivers average learning gains of 0.42 SD when taken up and 0.85 SD when implemented with high fidelity. To investigate how implementation can be improved in future settings, we run a new randomized trial of a targeted instruction program in Botswana. Results demonstrate that implementation can be improved in the context of a scaling program with large causal effects on learning. While research on implementation has been limited to date, our findings and framework reveal its importance for impact evaluation and generalizability. [Funding for this report was provided by the Center of Excellence for Development Impact and Learning (CEDIL).] (As Provided).
Anmerkungen	Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: AISR_Info@brown.edu; Web site: http://www.annenberginstitute.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01