A Comparison of Holistic, Analytic, and Part Marking Models in Speaking Assessment

Autor/inn/en	Khabbazbashi, Nahal; Galaczi, Evelina D.
Titel	A Comparison of Holistic, Analytic, and Part Marking Models in Speaking Assessment
Quelle	In: Language Testing, 37 (2020) 3, S.333-360 (28 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Khabbazbashi, Nahal)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532219898635
Schlagwörter	Holistic Approach; Classification; Grading; Language Tests; Speech Communication; Comparative Analysis; Scores; Guidelines; Rating Scales; Test Validity; Correlation; Second Language Learning; Evaluators; English (Second Language); Foreign Countries; Europe + Suchen Sie Ihr Suchwort? Holistischer Ansatz; Classification system; Klassifikation; Klassifikationssystem; Notengebung; Schulnote; Language test; Sprachtest; Richtlinien; Rating-Skala; Testvalidität; Korrelation; Zweitsprachenerwerb; English as second language; English; Second Language; Englisch als Zweitsprache; Ausland; Europa
Abstract	This mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings--which suggested stronger measurement properties for the part MM--phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. Although strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01