A Within-Study Approach to Evaluating the Role of Moderators of Impact in Limiting Generalizations from "Large to Small"

Autor/inn/en	Jaciw, Andrew P.; Unlu, Fatih; Nguyen, Thanh
Titel	A Within-Study Approach to Evaluating the Role of Moderators of Impact in Limiting Generalizations from "Large to Small"
Quelle	In: American Journal of Evaluation, 43 (2022) 1, S.108-131 (24 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Jaciw, Andrew P.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1098-2140
DOI	10.1177/10982140211030552
Schlagwörter	Program Effectiveness; Generalization; Outcome Measures; Institutional Characteristics; Class Size; Grade 2; Elementary School Students; Mathematics Achievement; Evaluation Methods; Tennessee + Suchen Sie Ihr Suchwort? Klassengröße; School year 02; 2. Schuljahr; Schuljahr 02; Mathmatics sikills; Mathmatics achievement; Mathematical ability; Mathematische Kompetenz
Abstract	There is a burgeoning body of evidence on the average impacts of educational programs. Yet, for many local decision makers, because impacts can vary across sites, the question of whether a certain program will work in their particular district or school remains. This article addresses the question of the generalizability of large-scale average causal effects to specific locales, that is, from "large to small." Our method evaluates whether impacts measured on the large-scale generalize to individual sites and whether adjusting for variation among sites in their characteristics accounts for impact heterogeneity that limits generalization. Our application to a multisite experimental evaluation in education shows that the average impact does not readily generalize to sites, with discrepancies between 0.17 and 0.41 standard deviations, and that characteristics of sites account only minimally for this variation. This emphasizes need for caution by local decision makers in accepting average impact findings as generalizable to their specific locales. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01