The Impact of World Englishes on Language Assessment: Rater Attitude, Rating Behavior, and Challenges

Literaturnachweis - Detailanzeige

Trefferliste

Autor/in	Hsu, Huei-Lien
Titel	The Impact of World Englishes on Language Assessment: Rater Attitude, Rating Behavior, and Challenges
Quelle	(2012), (287 Seiten) PDF als Volltext Verfügbarkeit Ph.D. Dissertation, University of Illinois at Urbana-Champaign
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
ISBN	978-1-3032-2006-7
Schlagwörter	Hochschulschrift; Dissertation; Language Variation; English (Second Language); Second Language Learning; Mixed Methods Research; Culture Fair Tests; Language Tests; Language Proficiency; Evaluators; Attitude Measures; Scoring; Test Validity; Language Teachers; Semantic Differential; Global Approach; Factor Analysis; Standard Spoken Usage; Indians; Computer Assisted Testing; Scores; Multivariate Analysis; Positive Attitudes; Negative Attitudes; Language Fluency; Evaluation Criteria; Sentence Structure; Vocabulary Development; Pronunciation; Language Attitudes; Foreign Countries; Multiple Regression Analysis; International English Language Testing System + Suchen Sie Ihr Suchwort? Thesis; Dissertations; Academic thesis; Sprachenvielfalt; English as second language; English; Second Language; Englisch als Zweitsprache; Zweitsprachenerwerb; Language test; Sprachtest; Language skill; Language skills; Sprachkompetenz; Bewertung; Testvalidität; Language teacher; Sprachunterricht; Globales Denken; Faktorenanalyse; Gesprochene Sprache; Umgangssprache; Inder; Multivariate Analyse; Negative Fixierung; Satzbau; Satzstruktur; Wortschatzarbeit; Aussprache; Sprachverhalten; Ausland; Language tests; Englisch
Abstract	By centralizing the issue of test fairness in language proficiency assessments, this study responds to a call by researchers for developing greater social responsibility in the language testing agenda. As inquiries into language attitude and psychology indicate, there is an underlying uncertainty pertaining to the validity of test use and score interpretation based on listeners' bias against for non-standard English and negative evaluation of such speakers. Of greater relevance in oral proficiency assessment is that listeners, that is, raters, transfer such attitudes to their scoring judgments. As an attempt to address this issue, this study investigates the scoring validity of the IELTS speaking test by examining its relationship in relation to a criterion designed to measure rater attitudes towards World Englishes. Validity arguments were formulated to guide two independent, yet related, studies based on mixed-methods approach and evaluate the claims and hypotheses set for the studies. In view of the lack of instruments measuring rater attitude towards global English in the language assessment context, the first study constructed the criterion measure, the Rater Attitude Instrument (RAI), involving 119 ESL teacher raters in the U.S. and India. As a result of the three-phase development, the RAI comprises 22 semantic differential scale items and 32 Likert scale items representing the three attitude dimensions of feeling, cognition, and behavior tendency used by psychologists. Confirmatory factor analysis supports the internal structure of the RAI with acceptable model fit indices (?[superscript 2] = 20.052, p = 0.094, RMSEA = 0.076, CFI = 0.954, TLI = 0.926). Content validity is ensured through teacher raters and content experts perspectives that continuously shaped the substance of the RAI. As the RAI demonstrates, rater attitudes towards World Englishes were generally positive and tended to focus more on speech comprehensibility; nevertheless, the majority of raters were inclined towards a preference for standard English in their scoring judgments. In the second study, the RAI and the six IELTS descriptive tasks produced by Indian examinees were administered on-line to the 96 teacher raters and the data analyzed to evaluate the extent to which the claim that rater attitude is a biasing factor affecting their scoring judgment on IELTS descriptive tasks can be sustained. The RAI scores were analyzed by FACETS that classified the raters into positive, neutral, and negative attitude groups according to measurement logit. Next, MANOVA was performed which suggested that the ratings by the positive and negative attitude groups were significantly different, with the positive group consistently rating higher on all the four criteria of Fluency, Sentence Structure, Vocabulary, and Pronunciation. The neutral and negative attitude groups rated significantly differently on Sentence Structure and Vocabulary, with the former rating higher than the negative group. Moderate to strong correlations ranging from 0.272 to 0.569 were observed between the RAI and the IELTS descriptive task scores. Multiple regression analysis revealed that RAI scores accounted for a significant amount of variance on the IELTS descriptive tasks sub- and total-scores, ranging from 17.5% to 32.4%. Moreover, the Indian/non-Indian variable was the only rater background characteristic investigated that significantly related to the rater feeling scores that formed one of the triplet attitude constructs, though contributing to only 10% of the score variance. Lastly, the verbal protocol study provided insightful information suggesting that raters with positive attitudes generally took into account the expected performance of language learners while some negative-attitude raters used the native speaker model as the underlying criterion for judgment. The impact of the findings on validity argument, test fairness, and rater trainings are also discussed. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.] (As Provided).
Anmerkungen	ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01