Effects of Visual Representations and Associated Interactive Features on Student Performance on National Assessment of Educational Progress (NAEP) Pilot Science Scenario-Based Tasks

Autor/inn/en	Durán, Richard P.; Zhang, Ting; Sañosa, David; Stancavage, Fran
Institution	American Institutes for Research (AIR)
Titel	Effects of Visual Representations and Associated Interactive Features on Student Performance on National Assessment of Educational Progress (NAEP) Pilot Science Scenario-Based Tasks
Quelle	(2020), (59 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	National Competency Tests; Computer Assisted Testing; Science Tests; Test Items; Middle School Students; Grade 8; Visual Aids; Student Attitudes; Comprehension; Navigation (Information Systems); Interaction; Multimedia Materials; Test Construction; Construct Validity; National Assessment of Educational Progress + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Middle school; Middle schools; Student; Students; Mittelschule; Mittelstufenschule; Schüler; Schülerin; School year 08; 8. Schuljahr; Schuljahr 08; Anschauungsmaterial; Schülerverhalten; Verstehen; Verständnis; Interaktion; Testaufbau
Abstract	The National Assessment of Educational Progress's (NAEP's) transition to an entirely digitally based assessment (DBA) began in 2017. As part of this transition, new types of NAEP items have begun to be developed that leverage the DBA environment to measure a wider range of knowledge and skills. These new item types include the science scenario-based tasks (SBTs) that are the focus of this report. The present project was one study in a program of research designed to gather evidence relevant to claims that students' performances on NAEP's DBA tasks are valid measures of constructs defined in the NAEP content frameworks. More specifically, the goal was to provide information that could help inform design considerations for construct-relevant use of visual and interactive features in future science items and tasks. The study involved a cognitive lab investigation of 31 eighth-grade students performing five science SBTs that had been used in the 2015 NAEP pilot assessment. The results of the study suggest that students in the small sample generally understood the visual and interactive features of SBTs as intended, suggesting that NAEP science SBT development procedures are working well. That said, the study also suggests that better formulated principles, or guidelines, for visual and interactive features should be developed and that quality control focused on those principles should be integrated into the critical path. [This study was commissioned by the NAEP Validity Studies (NVS) Panel, which was formed by the American Institutes for Research under contract with the National Center for Education Statistics.] (ERIC).
Anmerkungen	American Institutes for Research. 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5000; Fax: 202-403-5001; e-mail: inquiry@air.org; Web site: http://www.air.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01