The Impact of Inconsistent Responders to Mixed-Worded Scales on Inferences in International Large-Scale Assessments

Autor/inn/en	Steinmann, Isa; Sánchez, Daniel; van Laar, Saskia; Braeken, Johan
Titel	The Impact of Inconsistent Responders to Mixed-Worded Scales on Inferences in International Large-Scale Assessments
Quelle	In: Assessment in Education: Principles, Policy & Practice, 29 (2022) 1, S.5-26 (22 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Steinmann, Isa) ORCID (Braeken, Johan)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0969-594X
DOI	10.1080/0969594X.2021.2005302
Schlagwörter	Response Style (Tests); Test Items; Achievement Tests; Foreign Countries; International Assessment; Self Concept Measures; Reliability; Questionnaires; Elementary School Students; Scores; Test Reliability; Grade 4; Grade 6; Progress in International Reading Literacy Study; Trends in International Mathematics and Science Study + Suchen Sie Ihr Suchwort? Antwortverhalten; Test content; Testaufgabe; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; Reliabilität; Fragebogen; Testreliabilität; School year 04; 4. Schuljahr; Schuljahr 04; School year 06; 6. Schuljahr; Schuljahr 06
Abstract	Questionnaire scales that are mixed-worded, i.e. include both positively and negatively worded items, often suffer from issues like low reliability and more complex latent structures than intended. Part of the problem might be that some responders fail to respond consistently to the mixed-worded items. We investigated the prevalence and impact of inconsistent responders in 37 primary education systems participating in the joint PIRLS/TIMSS 2011 assessment. Using the mean absolute difference method and three mixed-worded self-concept scales, we identified between 2%-36% of students as inconsistent responders across education systems. Consistent with expectations, these students showed lower average achievement scores and had a higher risk of being identified as inconsistent on more than one scale. We also found that the inconsistent responders biased the estimated dimensionality and reliability of the scales. The impact on external validity measures was limited and unsystematic. We discuss implications for the use and development of questionnaire scales. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01