Effective Rating Scale Development for Speaking Tests: Performance Decision Trees

Autor/inn/en	Fulcher, Glenn; Davidson, Fred; Kemp, Jenny
Titel	Effective Rating Scale Development for Speaking Tests: Performance Decision Trees
Quelle	In: Language Testing, 28 (2011) 1, S.5-29 (25 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532209359514
Schlagwörter	Speech Communication; Rating Scales; Inferences; English (Second Language); Second Language Learning; Language Tests; Measurement; Language Usage; Validity; Pragmatics; Communicative Competence (Languages); Marketing; Service Occupations + Suchen Sie Ihr Suchwort? Rating-Skala; Inference; Inferenz; English as second language; English; Second Language; Englisch als Zweitsprache; Zweitsprachenerwerb; Language test; Sprachtest; Messverfahren; Sprachgebrauch; Gültigkeit; Pragmalinguistik; Communicative competence; Languages; Kommunikative Kompetenz; Sprache; Dienstleistungsberuf
Abstract	Rating scale design and development for testing speaking is generally conducted using one of two approaches: the measurement-driven approach or the performance data-driven approach. The measurement-driven approach prioritizes the ordering of descriptors onto a single scale. Meaning is derived from the scaling methodology and the agreement of trained judges as to the place of any descriptor on the scale. The performance data-driven approach, on the other hand, places primary value upon observations of language performance, and attempts to describe performance in sufficient detail to generate descriptors that bear a direct relationship with the original observations of language use. Meaning is derived from the link between performance and description. We argue that measurement-driven approaches generate impoverished descriptions of communication, while performance data-driven approaches have the potential to provide richer descriptions that offer sounder inferences from score meaning to performance in specified domains. With reference to original data and the literature on travel service encounters, we devise a new scoring instrument, a Performance Decision Tree (PDT). This instrument prioritizes what we term "performance effect" by explicitly valuing and incorporating performance data from a specific communicative context. We argue that this avoids the reification of ordered scale descriptors which we find in measurement-driven scale construction for speaking tests. (Contains 1 figure.) (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10