How Methodology Decisions Affect the Variability of Schools Identified as Beating the Odds. REL 2015-071

Autor/inn/en	Abe, Yasuyo; Weinstock, Phyllis; Chan, Vincent; Meyers, Coby; Gerdeman, R. Dean; Brandt, W. Christopher
Institution	National Center for Education Evaluation and Regional Assistance (ED); Regional Educational Laboratory Midwest (ED); American Institutes for Research
Titel	How Methodology Decisions Affect the Variability of Schools Identified as Beating the Odds. REL 2015-071
Quelle	(2015), (49 Seiten) PDF als Volltext (1); PDF als Volltext kostenfreie Datei (2) Verfügbarkeit
Zusatzinformation	Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	School Effectiveness; Identification; Statistical Analysis; Comparative Analysis; Prediction; Institutional Characteristics; Elementary Secondary Education; Academic Achievement; Public Schools; Michigan + Suchen Sie Ihr Suchwort? Schuleffizienz; Identifikation; Identifizierung; Statistische Analyse; Vorhersage; Schulleistung; Public school; Öffentliche Schule
Abstract	Schools that show better academic performance than would be expected given characteristics of the school and student populations are often described as "beating the odds" (BTO). State and local education agencies often attempt to identify such schools as a means of identifying strategies or practices that might be contributing to the schools' relative success. Key decisions on how to identify BTO schools may affect whether schools make the BTO list and thereby the identification of practices used to beat the odds. The purpose of this study was to examine how a list of BTO schools might change depending on the methodological choices and selection of indicators used in the BTO identification process. This study considered whether choices of methodologies and type of indicators affect the schools that are identified as BTO. The three indicators were (1) type of performance measure used to compare schools, (2) the types of school characteristics used as controls in selecting BTO schools, and (3) the school sample configuration used to pool schools across grade levels. The study applied statistical models involving the different methodologies and indicators and documented how the lists schools identified as BTO changed based on the models. Public school and student data from one midwest state from 2007-08 through 2010-11 academic years were used to generate BTO school lists. By performing pairwise comparisons among BTO school lists and computing agreement rates among models, the project team was able to gauge the variation in BTO identification results. Results indicate that even when similar specifications were applied across statistical methods, different sets of BTO schools were identified. In addition, for each statistical method used, the lists of BTO schools identified varied with the choice of indicators. Fewer than half of the schools were identified as BTO in more than one year. The results demonstrate that different technical decisions can lead to different identification results. Three appendices include: (1) Measures of classification accuracy; (2) Data, measures, and outliers and missing data; and (3) Classification tables. (As Provided).
Anmerkungen	Regional Educational Laboratory Midwest. Available from: Institute of Education Sciences. 555 New Jersey Avenue NW, Washington, DC 20208. Tel: 800-872-5327; Web site: http://ies.ed.gov/ncee/edlabs/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2022/4/11