The Effect of Ignoring Classroom-Level Variance in Estimating the Generalizability of School Mean Scores

Autor/inn/en	Wei, Xin; Haertel, Edward
Titel	The Effect of Ignoring Classroom-Level Variance in Estimating the Generalizability of School Mean Scores
Quelle	In: Educational Measurement: Issues and Practice, 30 (2011) 1, S.13-22 (10 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/j.1745-3992.2010.00196.x
Schlagwörter	Scores; Reliability; Computation; Generalizability Theory; Statistical Bias; Correlation; Error of Measurement; Accountability + Suchen Sie Ihr Suchwort? Reliabilität; Korrelation; Messfehler; Verantwortung
Abstract	Contemporary educational accountability systems, including state-level systems prescribed under No Child Left Behind as well as those envisioned under the "Race to the Top" comprehensive assessment competition, rely on school-level summaries of student test scores. The precision of these score summaries is almost always evaluated using models that ignore the classroom-level clustering of students within schools. This paper reports balanced and unbalanced generalizability analyses investigating the consequences of ignoring variation at the level of classrooms within schools when analyzing the reliability of such school-level accountability measures. Results show that the reliability of school means cannot be determined accurately when classroom-level effects are ignored. Failure to take between-classroom variance into account biases generalizability (G) coefficient estimates downward and standard errors (SEs) upward if classroom-level effects are regarded as fixed, and biases G-coefficient estimates upward and SEs downward if they are regarded as random. These biases become more severe as the difference between the school-level intraclass correlation (ICC) and the class-level ICC increases. School-accountability systems should be designed so that classroom (or teacher) level variation can be taken into consideration when quantifying the precision of school rankings, and statistical models for school mean score reliability should incorporate this information. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10