Examining Testlet Effects in the TestDaF Listening Section: A Testlet Response Theory Modeling Approach

Autor/in	Eckes, Thomas
Titel	Examining Testlet Effects in the TestDaF Listening Section: A Testlet Response Theory Modeling Approach
Quelle	In: Language Testing, 31 (2014) 1, S.39-61 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532213492969
Schlagwörter	Test Items; Language Tests; Listening Comprehension Tests; German; Second Languages; Item Response Theory; Test Reliability; Difficulty Level; Psychometrics; Foreign Students; College Applicants; Foreign Countries; Bulgaria; China; Germany; Russia; South Korea; Ukraine + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Language test; Sprachtest; Hörverstehensübung; Deutscher; Second language; Zweitsprache; Item-Response-Theorie; Testreliabilität; Schwierigkeitsgrad; Psychometry; Psychometrie; College applications; Studienbewerber; Ausland; Bulgarien; Deutschland; Russland; Korea; Republik
Abstract	Testlets are subsets of test items that are based on the same stimulus and are administered together. Tests that contain testlets are in widespread use in language testing, but they also share a fundamental problem: Items within a testlet are locally dependent with possibly adverse consequences for test score interpretation and use. Building on testlet response theory (Wainer, Bradlow, & Wang, 2007), the listening section of the Test of German as a Foreign Language (TestDaF) was analyzed to determine whether, and to which extent, testlet effects were present. Three listening passages (i.e., three testlets) with 8, 10, and 7 items, respectively, were analyzed using a two-parameter logistic testlet response model. The data came from two live exams administered in April 2010 ("N" = 2859) and November 2010 ("N" = 2214). Results indicated moderate effects for one testlet, and small effects for the other two testlets. As compared to a standard IRT analysis, neglecting these testlet effects led to an overestimation of test reliability and an underestimation of the standard error of ability estimates. Item difficulty and item discrimination estimates remained largely unaffected. Implications for the analysis and evaluation of testlet-based tests are discussed. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10