Comparing OECD PISA Reading in English to Other Languages: Identifying Potential Sources of Non-Invariance

Autor/inn/en	Asil, Mustafa; Brown, Gavin T. L.
Titel	Comparing OECD PISA Reading in English to Other Languages: Identifying Potential Sources of Non-Invariance
Quelle	In: International Journal of Testing, 16 (2016) 1, S.71-93 (23 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1530-5058
DOI	10.1080/15305058.2015.1064431
Schlagwörter	Comparative Analysis; Reading Achievement; Achievement Tests; Secondary School Students; Correlation; Foreign Countries; Culture Fair Tests; Criticism; Cultural Differences; Cross Cultural Studies; Reading Tests; Factor Analysis; Effect Size; Socioeconomic Status; Reading Comprehension; Test Construction; Error of Measurement; Educational Resources; Role; Educational Policy; Item Response Theory; Goodness of Fit; Australia; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Leseleistung; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Sekundarschüler; Korrelation; Ausland; Kritik; Kultureller Unterschied; Cultural comparison; Kulturvergleich; Lesetest; Faktorenanalyse; Socio-economic status; Sozioökonomischer Status; Leseverstehen; Testaufbau; Messfehler; Bildungsmittel; Rollen; Politics of education; Bildungspolitik; Item-Response-Theorie; Australien
Abstract	The use of the Programme for International Student Assessment (PISA) across nations, cultures, and languages has been criticized. The key criticisms point to the linguistic and cultural biases potentially underlying the design of reading comprehension tests, raising doubts about the legitimacy of comparisons across economies. Our research focused on the type and magnitude of invariance or non-invariance in the PISA Reading Comprehension test by language, culture, and economic development relative to performance of the Australian English-speaking reference group used to develop the tests. Multi-Group Confirmatory Factor Analysis based on means and covariance structure (MACS) modeling was used to establish a d[subscript MACS] effect size index for each economy for the degree of non-invariance. Only three wealthy, English-speaking countries had scalar invariance with Australia. Moderate or large effects were observed in just 31% of the comparisons. PISA index of economic, social and cultural status had a moderate inverse correlation with dMACS suggesting that socioeconomic resourcing of education played a significant role in measurement invariance, while educational practice and language factors seemed to play a further small role in non-invariance. Alternative approaches to reporting PISA results consistent with non-invariance are suggested. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01