Beurteilerübereinstimmung und schwer zu beurteilende Texte im Vergleich Training und Bildungsstandardüberprüfung in Deutsch auf der vierten Schulstufe.

Autor/inn/en	Freunberger, Roman; Breit, Simone; Illetschko, Marcel
Titel	Beurteilerübereinstimmung und schwer zu beurteilende Texte im Vergleich Training und Bildungsstandardüberprüfung in Deutsch auf der vierten Schulstufe.
Quelle	Aus: Sigott, Günther (Hrsg.): Language testing in Austria. Taking stock. Berlin u.a.: Peter Lang (2018) S. 373-388 PDF als Volltext Link als defekt meldenVerfügbarkeit
Reihe	Language testing and evaluation. 40
Sprache	deutsch
Dokumenttyp	online; gedruckt; Sammelwerksbeitrag
ISBN	978-3-631-74938-8; 978-3-631-75973-8; 978-3-631-75974-5; 978-3-631-75975-2
Schlagwörter	Vergleichende Erziehungswissenschaft; Bildungsforschung; Validität; Sprachtest; Bestandsaufnahme; Österreich
Abstract	The Austrian standards-based proficiency test for 4th graders in German language comprises, amongst tests of other sub-competencies, an assessment of students' written performances. The short texts are judged by expert raters who underwent highly standardized training which involved benchmark texts. As raters' judgements are prone to rater bias or rater error such as leniency/severity or inaccuracy, monitoring of rater behavior during training is mandatory to ensure reliable test scores in the field test. Previous research has tended to focus on pure rater effects. Here, we would like to establish a link to other findings that showed that rater inaccuracy seems not to be constant over texts. In fact, some texts are harder to rate than others. Following this line of reasoning we apply an accuracy index to our training data which can identify hard-to-rate texts. Also, we tackle the question of the consistency of rater agreement between training and field test. Our findings provide evidence that rating quality slightly decreases over time and that some texts give rise to difficulties during the rating process which might be due to text features, which future studies should focus on. The findings are discussed with respect to their relevance for improving rater training. (Verlag).
Erfasst von	Externer Selbsteintrag
Update	2021/3