Re-Examining the Content Validation of a Grammar Test: The (Im)Possibility of Distinguishing Vocabulary and Structural Knowledge

Autor/inn/en	Alderson, J. Charles; Kremmel, Benjamin
Titel	Re-Examining the Content Validation of a Grammar Test: The (Im)Possibility of Distinguishing Vocabulary and Structural Knowledge
Quelle	In: Language Testing, 30 (2013) 4, S.535-556 (22 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0265-5322
DOI	10.1177/0265532213489568
Schlagwörter	Grammar; Language Tests; Content Validity; Test Validity; Syntax; Vocabulary; Reading Ability; Expertise; Second Languages; Content Analysis; Replication (Evaluation) + Suchen Sie Ihr Suchwort? Grammatik; Language test; Sprachtest; Testvalidität; Wortschatz; Reading competence; Lesekompetenz; Expert appraisal; Second language; Zweitsprache; Inhaltsanalyse
Abstract	"Vocabulary and structural knowledge" (Grabe, 1991, p. 379) appears to be a key component of reading ability. However, is this component to be taken as a unitary one or is structural knowledge a separate factor that can therefore also be tested in isolation in, say, a test of syntax? If syntax can be singled out (e.g. in order to investigate its contribution to reading ability), this test of syntactic knowledge would require validation. The usefulness and reliability of using expert judgments as a means of analysing the content or difficulty of test items in language assessment has been questioned for more than two decades. Still, groups of expert judges are often called upon as they are perceived to be the only or at least a very convenient way of establishing key features of items. Such judgments, however, are particularly opaque and thus problematic when judges are required to make categorizations where categories are only vaguely defined or are ontologically questionable in themselves. This is, for example, the case when judges are asked to classify the content of test items based on a distinction between lexis and syntax, a dichotomy corpus linguistics has suggested cannot be maintained. The present paper scrutinizes a study by Shiotsu (2010) that employed expert judgments, on the basis of which claims were made about the relative significance of the components "syntactic knowledge" and "vocabulary knowledge" in reading in a second language. By both replicating and partially replicating Shiotsu's (2010) content analysis study, the paper problematizes not only the issue of the use of expert judgments, but, more importantly, their usefulness in distinguishing between construct components that might, in fact, be difficult to distinguish anyway. This is particularly important for an understanding and diagnosis of learners' strengths and weaknesses in reading in a second language. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10