A Validity Study of the NAEP Full Population Estimates

Autor/inn/en	Hedges, Larry V.; Bandeira de Mello, Victor
Institution	National Center for Education Statistics (ED); American Institutes for Research
Titel	A Validity Study of the NAEP Full Population Estimates
Quelle	(2013), (47 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	National Competency Tests; Test Validity; Inclusion; Statistical Significance; Achievement Gains; Mathematics Achievement; Reading Achievement; Writing Achievement; Science Achievement; Testing Accommodations; Calculators; Test Format; Disabilities; National Assessment of Educational Progress + Suchen Sie Ihr Suchwort? Testvalidität; Inklusion; Achievement gain; Leistungssteigerung; Mathmatics sikills; Mathmatics achievement; Mathematical ability; Mathematische Kompetenz; Leseleistung; Testing process; Accessibility (for disabled); Accessibility; Disabled person; Testdurchführung; Testen; Barrierefreiheit; Zugänglichkeit; Behinderter; Calculator; Rechner; Rechenmaschine; Testentwicklung; Handicap; Behinderung
Abstract	In early 2001, to support an internal evaluation of the impact of changing exclusion rates on reports of statistically significant gains across states, the National Center for Education Statistics (NCES) sponsored research on imputation procedures of National Assessment of Educational Progress (NAEP) scores for the excluded students and provided adjusted or full population estimates (FPEs) for the 1996 to 2000 NAEP mathematics gains. The same method was subsequently used to produce FPEs for Grades 4 and 8 in reading, writing, mathematics, and science for each year these assessments were administered since 1994. In spite of considerable prior research, NAEP FPEs have not yet achieved operational status, and further validity research has been recommended. The occasion of two special inclusion studies in the 2011 NAEP assessment--designed to determine if students who are excluded from the operational assessment can meaningfully participate if offered one of two special booklets--presented just such an opportunity for additional validity research on the FPEs. The Educational Testing Service (ETS) analyzed the data from these otherwise excluded students, but the analyses were not carried out for the purposes of validating the FPEs. The purpose of this document is to report a study that capitalized on these inclusion studies to obtain further validity evidence about the FPEs and to better inform a possible decision to move toward operational status for the FPE methodology. Fieldwork for two closely related inclusion studies was carried out as part of the 2011 NAEP Mathematics Assessment. The two studies had slightly different student populations and examined different accommodations. One study focused on the use of calculators as an accommodation (in states that permit this accommodation on their state assessments). The other study focused on an inclusion booklet composed of Knowledge and Skill Appropriate (KaSA) blocks. In some states, there were students included in both studies (i.e., some students included because of the calculator accommodation and other students included because of the inclusion blocks). In other states, only the inclusion block was offered because the states did not allow a calculator accommodation on their state assessments. In conclusion, these studies suggest that the calculator block and KaSA booklet accommodations, by themselves, will not change the number of included students enough to have a substantial impact on national parameter estimates. Each study is outlined briefly in this report. Procedures for Calculating Full Population Estimates is appended. (ERIC).
Anmerkungen	American Institutes for Research. 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5000; Fax: 202-403-5001; e-mail: inquiry@air.org; Web site: http://www.air.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10