A Generalizability Theory Study of Optimal Measurement Design for a Summative Assessment of English/Chinese Consecutive Interpreting

Autor/in	Han, Chao
Titel	A Generalizability Theory Study of Optimal Measurement Design for a Summative Assessment of English/Chinese Consecutive Interpreting
Quelle	In: Language Testing, 36 (2019) 3, S.419-438 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Han, Chao)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532218809396
Schlagwörter	Generalization; Decision Making; Summative Evaluation; Evaluators; Language Fluency; Chinese; English; English (Second Language); Scores; Correlation; Translation; Measurement; Language Tests; Program Development; Second Language Learning; Second Language Instruction; Foreign Countries; Undergraduate Students; Task Analysis; Evaluation Criteria; China + Suchen Sie Ihr Suchwort? Decision-making; Entscheidungsfindung; Language skill; Language skills; Sprachkompetenz; China; Chinesen; English language; Englisch; English as second language; English; Second Language; Englisch als Zweitsprache; Korrelation; Messverfahren; Language test; Sprachtest; Programmplanung; Zweitsprachenerwerb; Fremdsprachenunterricht; Ausland; Aufgabenanalyse
Abstract	Summative assessment of interpretation is widely conducted in interpreting courses/programs to inform high-stakes decision making, such as the selection, certification, and conferral of academic degrees. Yet there has been very limited empirical research to investigate the score dependability of summative interpretation assessment. The present study therefore sets out to explore the optimal measurement design(s) for a locally created summative assessment of English/Chinese consecutive interpretation, based on multiple fully crossed generalizability studies. Major findings include the following: (a) overall the raters behaved more consistently by using the information completeness (InfoCom) scale rather than the fluency of delivery (FluDel) or target language quality (TLQual) scales; (b) the raters displayed greater variability in evaluating the Chinese-to-English interpretation rather than the English-to-Chinese interpretation; (c) although adding tasks worked more effectively in raising score dependability than using additional raters for the InfoCom ratings in the English-to-Chinese interpretation, the pattern was reversed for the other observations; and (d) two potentially optimal designs were identified for the English-to-Chinese direction, and one design for the other direction. These results are discussed, highlighting the complex nature of relationships among the assessment criterion, the interpreting directionality, the raters' dominant language and score dependability, together with the need to ensure score dependability for summative interpretation assessment. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01