Question Preview in English for Academic Purposes Listening Assessment: The Effect of Stem Preview on Difficulty, Item Type, and Discrimination

Autor/inn/en	Yeager, Rebecca; Meyer, Zachary
Titel	Question Preview in English for Academic Purposes Listening Assessment: The Effect of Stem Preview on Difficulty, Item Type, and Discrimination
Quelle	In: International Journal of Listening, 36 (2022) 3, S.299-324 (26 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Yeager, Rebecca) ORCID (Meyer, Zachary)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1090-4018
DOI	10.1080/10904018.2022.2029705
Schlagwörter	English for Academic Purposes; Second Language Learning; Second Language Instruction; Teaching Methods; Multiple Choice Tests; Listening Comprehension Tests; Test Items; Item Analysis; Scores; Difficulty Level; Test Format; Test Preparation; Undergraduate Students; English (Second Language); Language Tests; Lecture Method; Economics Education; Architectural Education; Comparative Analysis; Test of English as a Foreign Language + Suchen Sie Ihr Suchwort? Zweitsprachenerwerb; Fremdsprachenunterricht; Teaching method; Lehrmethode; Unterrichtsmethode; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Hörverstehensübung; Test content; Testaufgabe; Itemanalyse; Schwierigkeitsgrad; Testentwicklung; English as second language; English; Second Language; Englisch als Zweitsprache; Language test; Sprachtest; Wirtschaftskunde
Abstract	This study investigates the effects of adding stem preview to an English for Academic Purposes (EAP) multiple-choice listening assessment. In stem preview, listeners may view the item stems, but not response options, before listening. Previous research indicates that adding preview to an exam typically decreases difficulty, but raises concerns about score interpretation. Concerningly, no previous studies have explored the impact of preview on item discrimination, a key assumption of a validity argument. Our study utilized a Latin square design controlling for group, lecture, and preview condition to explore the impact of stem preview on difficulty, item type, and discrimination. Analysis indicated no significant effects of preview condition on difficulty or item type overall at our chosen alpha level. However, comparisons of total scores revealed a bimodal distribution in the no-preview condition, but not in the preview condition, indicating lower-scoring students received a boost from stem preview. Additionally, preview significantly increased facility on one of the trivial items. Results for discrimination were more complicated. On two of the five discrimination indices in the study, preview significantly decreased item discrimination, but for the other three indices, effects were not significant. Implications for assessment developers and researchers are discussed. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01