Does It Matter if I Take My Writing Test on Computer? An Empirical Study of Mode Effects in NAEP

Autor/inn/en	Horkay, Nancy; Bennett, Randy Elliott; Allen, Nancy; Kaplan, Bruce; Yan, Fred
Titel	Does It Matter if I Take My Writing Test on Computer? An Empirical Study of Mode Effects in NAEP
Quelle	In: Journal of Technology, Learning, and Assessment, 5 (2006) 2, (50 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1540-2525
Schlagwörter	Writing Evaluation; Writing Tests; Computer Assisted Testing; Program Effectiveness; Grade 8; Writing Skills; Correlation; Comparative Analysis; Educational Assessment; Elementary School Students; Test Items; Educational Testing; Student Evaluation; Internet; Educational Technology; Test Format; Test Content; Comparative Testing; Printed Materials; Methods Research; Test Bias; Laptop Computers; Computer Literacy + Suchen Sie Ihr Suchwort? Writing test; Schreibtest; School year 08; 8. Schuljahr; Schuljahr 08; Writing skill; Schreibfertigkeit; Korrelation; Education; assessment; Bewertungssystem; Test content; Testaufgabe; Schulnote; Studentische Bewertung; Unterrichtsmedien; Testentwicklung; Methodenforschung; Testkritik; Laptop computer; Laptop; Computer; Digitalrechner; Computerkenntnisse
Abstract	This study investigated the comparability of scores for paper and computer versions of a writing test administered to eighth grade students. Two essay prompts were given on paper to a nationally representative sample as part of the 2002 main NAEP writing assessment. The same two essay prompts were subsequently administered on computer to a second sample also selected to be nationally representative. Analyses looked at overall differences in performance between the delivery modes, interactions of delivery mode with group membership, differences in performance between those taking the computer test on different types of equipment (i.e., school machines vs. NAEP-supplied laptops), and whether computer familiarity was associated with online writing test performance. Results generally showed no significant mean score differences between paper and computer delivery. However, computer familiarity significantly predicted online writing test performance after controlling for paper writing skill. These results suggest that, for any given individual, a computer-based writing assessment may produce different results than a paper one, depending upon that individual's level of computer familiarity. Further, for purposes of estimating population performance, as long as substantial numbers of students write better on computer than on paper (or better on paper than on computer), conducting a writing assessment in either mode alone may underestimate the performance that would have resulted if students had been tested using the mode in which they wrote best. (Contains 13 endnotes, 14 tables, and 2 figures.) (As Provided).
Anmerkungen	Technology and Assessment Study Collaborative. 332 Campion Hall, Boston College, Chestnut Hill, MA 02467. Tel: 617-552-4521; Fax: 617-552-8419; e-mail: jtal_editor@bc.edu; Web site: http://escholarship.bc.edu/jtla
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10