Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis

Autor/in	Huang, Chiungjung
Titel	Magnitude of Task-Sampling Variability in Performance Assessment: A Meta-Analysis
Quelle	In: Educational and Psychological Measurement, 69 (2009) 6, S.887-912 (26 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/0013164409344550
Schlagwörter	Test Bias; Research Design; Performance Based Assessment; Performance Tests; Sampling; Test Items; Meta Analysis; Task Analysis; Difficulty Level; Second Language Learning; Mathematics Skills; Listening Skills; Effect Size; Interrater Reliability; Scoring; Coding + Suchen Sie Ihr Suchwort? Testkritik; Forschungsdesign; Leistungsermittlung; Leistungsbeurteilung; Leistungsmessung; Leistungsüberprüfung; Test content; Testaufgabe; Meta-analysis; Metaanalyse; Aufgabenanalyse; Schwierigkeitsgrad; Zweitsprachenerwerb; Mathmatics achievement; Mathematics ability; Mathematische Kompetenz; Interrater-Reliabilität; Bewertung; Codierung; Programmierung
Abstract	This study examined the percentage of task-sampling variability in performance assessment via a meta-analysis. In total, 50 studies containing 130 independent data sets were analyzed. Overall results indicate that the percentage of variance for (a) differential difficulty of task was roughly 12% and (b) examinee's differential performance of the same task was approximately 26%. Based on moderator analysis, research design and subject area were significant predictors of variance component for tasks ([sigma][superscript 2][subscript t]) and variance component for person-task interaction ([sigma][superscript 2][subscript pt]). Mathematics and listening were associated with relatively high proportion of task-sampling variability whereas foreign language was least affected by this variability. To reduce task variation, some techniques are recommended: (a) incorporating more facets when possible, (b) using a crossed rather than a nested design, and (c) incorporating occasion as a facet. (Contains 3 tables.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10