Scoring with Nominal Missing-Response Parameters: Its Effects on Achievement Levels Set on the National Assessment of Educational Progress (NAEP).

Autor/inn/en	Bay, Luz; Loomis, Susan Cooper; Wang, Tianyou
Titel	Scoring with Nominal Missing-Response Parameters: Its Effects on Achievement Levels Set on the National Assessment of Educational Progress (NAEP).
Quelle	(1995), (43 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Academic Achievement; Comparative Analysis; Cutting Scores; High School Seniors; High Schools; Item Response Theory; National Competency Tests; Probability; Scoring; Test Results; National Assessment of Educational Progress + Suchen Sie Ihr Suchwort? Schulleistung; High school; Oberschule; Item-Response-Theorie; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Bewertung
Abstract	This study examines the effects on the National Assessment of Educational Progress (NAEP) achievement levels of using item response theory (IRT) models that have nominal missing-response parameters. It compares cutpoints based on item parameters that were fitted using two different models. The first set of cutpoints were based on parameters for the two- and three-parameter logistic model, and the second set of cutpoints were based on R. Bock's (1972) nominal model. Data are from the 1992 NAEP in mathematics and reading for grade 12. For reading, data included 1,966 responses to a block of items, and for mathematics, data included 2,192 responses. Other data were the item-by-item ratings by each panelist who participated in the Achievement Levels Setting process for NAEP 1992. For each subject, the cutpoints set using the probability curves obtained by the different models were compared. The percent of students scoring at each level were also compared. For reading, the logistic model, when fitted to the data, converged in 25 iterations and yielded a marginal reliability of 0.67 with maximum information of 4.3 at theta equals -0.5. The nominal model converged in 88 iterations and yielded a marginal reliability of 0.85 with a maximum information of 14.2 at theta equals 0. For mathematics, the logistic model converged in 184 iterations and yielded a marginal reliability of 0.61 with maximum information of 23.9 at theta equals 1.5. The nominal model, with mathematics data, converged in 46 iterations, and yielded a marginal reliability of 0.62, with maximum information of 7.3 at theta equals -0.2. When the percentages of students scoring at or above each cutpoint were compared, none scored at the Advanced level using the nominal model. These preliminary results suggest the direction of future studies, but cannot be generalized to the NAEP assessment program. An appendix contains achievement levels descriptions. (Contains three tables, six figures, and five references.) (SLD)
Erfasst von	ERIC (Education Resources Information Center), Washington, DC