Literaturnachweis - Detailanzeige
| Autor/inn/en | Ling, Guangming; Mollaun, Pamela; Xi, Xiaoming |
|---|---|
| Titel | A Study on the Impact of Fatigue on Human Raters When Scoring Speaking Responses |
| Quelle | In: Language Testing, 31 (2014) 4, S.479-499 (21 Seiten)Infoseite zur Zeitschrift
PDF als Volltext |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Zeitschriftenaufsatz |
| ISSN | 0265-5322 |
| DOI | 10.1177/0265532214530699 |
| Schlagwörter | Evaluators; Fatigue (Biology); Scoring; Performance; Audio Equipment; Language Tests; English (Second Language); Second Language Learning; Computer Assisted Testing; Accuracy; Test Validity; Oral Language; Test of English as a Foreign Language |
| Abstract | The scoring of constructed responses may introduce construct-irrelevant factors to a test score and affect its validity and fairness. Fatigue is one of the factors that could negatively affect human performance in general, yet little is known about its effects on a human rater's scoring quality on constructed responses. In this study, we compared the scoring quality of 72 raters under four shift conditions differing on the shift length (total scoring time in a day) and session length (time continuously spent on a task). About 14,000 audio responses to four TOEFL iBT speaking tasks were scored, including 5446 validity responses that have pre-assigned "true" scores used to measure scoring accuracy. Our results suggest that the overall scoring accuracy is high for the TOEFL iBT Speaking Test, but varying levels of rating accuracy and consistency exist across shift conditions. The raters working the shorter shifts or shorter sessions on average maintain greater rating productivity, accuracy, and consistency than those working longer shifts or sessions do. The raters working the 6-hour shift with three 2-hour sessions outperform those under other shift conditions in both rating accuracy and consistency. (As Provided). |
| Anmerkungen | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2017/4/10 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
