An Investigation of the Impact of Jagged Profile on L2 Speaking Test Ratings: Evidence from Rating and Eye-Tracking Data

Autor/inn/en	Ma, Wenyue; Winke, Paula
Titel	An Investigation of the Impact of Jagged Profile on L2 Speaking Test Ratings: Evidence from Rating and Eye-Tracking Data
Quelle	In: Language Assessment Quarterly, 19 (2022) 4, S.394-421 (28 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Ma, Wenyue) ORCID (Winke, Paula)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1543-4303
DOI	10.1080/15434303.2022.2078720
Schlagwörter	Language Tests; Second Language Learning; Speech Communication; Profiles; Scoring; Correlation; Holistic Approach; Eye Movements; Rating Scales; Pronunciation; Grammar; Schemata (Cognition); Training; Evaluators; Graduate Students; State Universities; English (Second Language); Native Speakers; Gender Differences; Teaching Assistants; Michigan + Suchen Sie Ihr Suchwort? Language test; Sprachtest; Zweitsprachenerwerb; Charakterisierung; Profilanalyse; Bewertung; Korrelation; Holistischer Ansatz; Augenbewegung; Rating-Skala; Aussprache; Grammatik; Cognition; Schema; Kognition; Ausbildung; Graduate Study; Student; Students; Aufbaustudium; Graduiertenstudium; Hauptstudium; Studentin; Staatliche Universität; English as second language; English; Second Language; Englisch als Zweitsprache; Muttersprachler; Geschlechterkonflikt
Abstract	The factors that influence rater scoring have been a subject of great interest to researchers in second language assessment. However, the research on the impact of test-takers' speech profiles (e.g., a jagged or a flat profile reflecting analytic subscores) on raters' scoring behaviors remains to be seen. To investigate the role of speech profiles in scoring, we collected analytic and holistic rating scores from 28 trained raters while they were marking the performances of three groups of speakers with distinct profiles, determined by prior ratings. We tracked eleven of the raters' eye-movements to record how often and how long they looked at the various categories on the rating scales. We found that the raters perceived speakers who have better pronunciation as overall more competent speakers. Meanwhile, speakers' score profiles influenced raters' attention: raters fixated longer and more often, and made more eye-visits, to the lexical grammar category while assessing speakers with a jagged profile. Raters spent less time assessing the pronunciation of the speakers who were pre-identified as having better pronunciation. The findings shed light on the impact of speech characteristics on raters' cognition and score assignments and therefore have important implications for rater training in L2 speaking assessments. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01