Variance Estimation in Evaluations with No-Shows: A Comparison of Methods

Autor/inn/en	Litwok, Daniel; Peck, Laura R.
Titel	Variance Estimation in Evaluations with No-Shows: A Comparison of Methods
Quelle	In: American Journal of Evaluation, 40 (2019) 1, S.104-118 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1098-2140
DOI	10.1177/1098214017749318
Schlagwörter	Computation; Outcomes of Treatment; Program Evaluation; Scaling; Experimental Groups; Control Groups; Error of Measurement; Intention; Statistical Bias; Compliance (Psychology) + Suchen Sie Ihr Suchwort? Programme evaluation; Programmevaluation; Scale construction; Skalenkonstruktion; Messfehler
Abstract	In experimental evaluations of policy interventions, the so-called Bloom adjustment is commonly used to estimate the impact of the treatment on the treated. It does so by rescaling the estimated impact of the intention to treat--that is, the overall treatment-control group difference in outcomes for the entire experimental sample--by the percentage of cases that took up the treatment offer. The practice of also rescaling the variance, as is common in the literature, imposes simplifying assumptions that may lead to biased variance estimates. We compare variances using the Bloom adjustment to variances that capture all the estimation error. While the difference between these variances is negligible in three experimental evaluations, we highlight three conditions that could result in larger relative bias: large impacts, large variability in compliance, and/or substantial endogeneity bias. The presence of these conditions could potentially result in a different conclusion for policy or practice. [An earlier version of this article was prepared, in collaboration with Luke Keele, with federal funding from the U.S. Department of Labor, Employment and Training Administration, under Contract Number GS10F0086J/DOLF091A21626.] (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01