Analyzing Regression-Discontinuity Designs with Multiple Assignment Variables: A Comparative Study of Four Estimation Methods

Literaturnachweis - Detailanzeige

Trefferliste

Autor/inn/en	Wong, Vivian C.; Steiner, Peter M.; Cook, Thomas D.
Institution	Society for Research on Educational Effectiveness (SREE)
Titel	Analyzing Regression-Discontinuity Designs with Multiple Assignment Variables: A Comparative Study of Four Estimation Methods
Quelle	(2012), (9 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Regression (Statistics); Comparative Analysis; Computation; Statistical Analysis + Suchen Sie Ihr Suchwort? Regression; Regressionsanalyse; Statistische Analyse
Abstract	In a traditional regression-discontinuity design (RDD), units are assigned to treatment and comparison conditions solely on the basis of a single cutoff score on a continuous assignment variable. The discontinuity in the functional form of the outcome at the cutoff represents the treatment effect, or the average treatment effect at the cutoff. However, units are often assigned to treatment on more than one continuous assignment variable. Recent applications of RD designs in education have had multiple assignment variables and cutoff scores available for treatment assignment. For example, Jacob and Lefgren (2004a) and Matsudaira (2008) examined the effects of summer remedial education programs that were assigned to students based on missing a reading score cutoff, a math cutoff or both. Kane (2003) and van der Klaauw (2002) evaluated the effects of college financial aid offers on students' postsecondary school attendance by using measures of income, assets and grade point average (Kane, 2003) or grade point average and SAT scores (van der Klaauw, 2002) as multiple assignment variables in an RD design. Papay, Murnane, and Willett (2010) and Martorell (2004) looked at the effects of failing high school exit exams in two subject areas--English language arts and math--on the probability of students' graduating from high school. Finally, Gill et al. (2007) examined the effects of schools' failure to make Adequate Yearly Progress (AYP) under No Child Left Behind by missing one of 39 possible assignment criteria. All are examples of the multivariate regression discontinuity design (MRDD), where treatment assignment is based on cutoffs for two or more covariates rather than a single point along an assignment variable. MRDDs are not unique to education; they also occur with increasing frequency in other fields of research, such as in the evaluation of labor market programs (Card, Chetty & Weber, 2007; Lalive, Van Ours & Zweimuller, 2006; Lalive, 2008). This paper has three purposes. The first is to use potential outcomes notation (Holland, 1986; Rubin, 1974) to define the causal estimand tau[subscript MORD] for an MRDD with two assignment variables (M and R) and cutoffs. The second purpose of this paper is to provide guidance on the complexities of choosing an appropriate causal estimand of interest. Finally, the paper seeks to test four analytic approaches for estimating treatment effects in an MRD design--the frontier, centering, univariate, and instrumental variable (IV) approaches--and to identify the causal estimand(s) produced by each approach. The authors' analytic and simulation will work highlight the complexities of choosing an appropriate causal estimand in an MRD design. In many cases, the frontier average treatment effect may not have a meaningful interpretation because it does not make sense to pool effects across multiple frontiers. If at one frontier, the estimate indicates no effect and at the other frontier, a significant positive effect, then the average effect across the entire frontier rests on a scale-dependent weighting scheme. In these cases, the authors recommend that researchers estimate frontier-specific effects because tau[subscript M] and tau[subscript R] can provide at least upper and lower bounds for the overall treatment effect. In addition, without strong assumptions (e.g., constant treatment effects), the frontier-specific effects tau[subscript M] and tau[subscript R] is less than general than what would be obtained from a traditional univariate RDD with a corresponding assignment variable and cutoff. That is because the cutoff of a traditional RDD is not restricted by the cutoffs of additional assignment variables. Still, the presence of multiple cutoff-frontiers has the advantage of exploring the heterogeneity of treatment effects along different dimensions. Finally, the frontier-specific and frontier average treatment effect cannot be generalized beyond the sub-population of units that is close to the cutoff frontiers. As with standard RDD, MRDD produces only the treatment effects along the cutoff frontier(s) as opposed to across the entire response surface. Thus, researchers have the onus of communicating to practitioners and policy-makers which causal quantities are evaluated, explaining why these are the causal quantities of interest, and discussing the benefits and limitations of the results. (Contains 1 figure.) (ERIC).
Anmerkungen	Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10