Evaluating the Risk of Nonresponse Bias in Educational Large-Scale Assessments with School Nonresponse Questionnaires: A Theoretical Study

Literaturnachweis - Detailanzeige

Trefferliste

Autor/inn/en	Meinck, Sabine; Cortes, Diego; Tieck, Sabine
Titel	Evaluating the Risk of Nonresponse Bias in Educational Large-Scale Assessments with School Nonresponse Questionnaires: A Theoretical Study
Quelle	In: Large-scale Assessments in Education, 5 (2017), Artikel 3 (21 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Meinck, Sabine)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2196-0739
DOI	10.1186/s40536-017-0038-6
Schlagwörter	Response Rates (Questionnaires); Bias; Educational Assessment; Institutional Characteristics; Academic Achievement; Correlation; Multivariate Analysis; Achievement Tests; Foreign Countries; Grade 4; Reading Achievement; International Assessment; Reading Tests; Elementary Secondary Education; Mathematics Tests; Mathematics Achievement; Science Tests; Science Achievement; Grade 8; Regression (Statistics); Statistical Analysis; Progress in International Reading Literacy Study; Trends in International Mathematics and Science Study + Suchen Sie Ihr Suchwort? Antwortkontrolle; Education; assessment; Bewertungssystem; Schulleistung; Korrelation; Multivariate Analyse; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; School year 04; 4. Schuljahr; Schuljahr 04; Leseleistung; Lesetest; Mathmatics sikills; Mathmatics achievement; Mathematical ability; Mathematische Kompetenz; School year 08; 8. Schuljahr; Schuljahr 08; Regression; Regressionsanalyse; Statistische Analyse
Abstract	Survey participation rates can have a direct impact on the validity of the data collected since nonresponse always holds the risk of bias. Therefore, the International Association for the Evaluation of Educational Achievement (IEA) has set very high standards for minimum survey participation rates. Nonresponse in IEA studies varies between studies and cycles. School participation is at a higher risk relative to within-school participation; school students are more likely to cooperate than adults (i.e., university students or school teachers). Across all studies conducted by the IEA during the last decade, between 7 and 33% of participating countries failed to meet the minimum participation rates at the school level. Quantifying the bias introduced by nonresponse is practically impossible with the currently implemented design. During the last decade social researchers have introduced and developed the concept of nonresponse questionnaires. These are shortened instruments applied to nonrespondents, and aim to capture information that correlates with both: survey's main outcome variable(s), and respondent's propensity of participation. We suggest in this paper a method to develop such questionnaires for nonresponding schools in IEA studies. By these means, we investigated school characteristics that are associated with students' average achievement scores using correlational and multivariate regression analysis in three recent IEA studies. We developed regression models that explain with only 11 school questionnaire variables or less up to 77% of the variance of the school mean achievement score. On average across all countries, the R[superscript 2] of these models was 0.24 (PIRLS), 0.34 (TIMSS, grade 4) and 0.36 (TIMSS grade 8), using 6-11 variables. We suggest that data from such questionnaires can help to evaluate bias risks in an effective way. Further, we argue that for countries with low participation rates a change in the approach of computing nonresponse adjustment factors to a system were school´s participation propensity determines the nonresponse adjustment factor should be considered. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01