A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator

Autor/inn/en	Smyk, Magdalena; Tyrowicz, Joanna; van der Velde, Lucas
Titel	A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator
Quelle	In: Sociological Methods & Research, 50 (2021) 1, S.429-464 (36 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0049-1241
DOI	10.1177/0049124118782538
Schlagwörter	Online Surveys; Data; Reliability; Wages; Income; Work Environment; Regression (Statistics); Computation; Probability; Statistical Analysis; Statistical Distributions; Statistical Bias; Foreign Countries + Suchen Sie Ihr Suchwort? Daten; Reliabilität; Wage; Löhne; Einkommen; Arbeitsmilieu; Regression; Regressionsanalyse; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Statistische Analyse; Wahrscheinlichkeitsverteilung; Ausland
Abstract	We investigate the reliability of data from the Wage Indicator (WI), the largest online survey on earnings and working conditions. Comparing WI to nationally representative data sources for 17 countries reveals that participants of WI are not likely to have been representatively drawn from the respective populations. Previous literature has proposed to utilize weights based on inverse propensity scores, but this procedure was shown to leave reweighted WI samples different from the benchmark nationally representative data. We propose a novel procedure, building on covariate balancing propensity score, which achieves complete reweighting of the WI data, making it able to replicate the structure of nationally representative samples on observable characteristics. While rebalancing assures the match between WI and representative benchmark data sources, we show that the wage schedules remain different for a large group of countries. Using the example of a Mincerian wage regression, we find that in more than a third of the cases, our proposed novel reweighting assures that estimates obtained on WI data are not biased relative to nationally representative data. However, in the remaining 60 percent of the analyzed 95 data sets, systematic differences in the estimated coefficients of the Mincerian wage regression between WI and nationally representative data persist even after reweighting. We provide some intuition about the reasons behind these biases. Notably, objective factors such as access to the Internet or richness appear to matter, but self-selection (on unobservable characteristics) among WI participants appears to constitute an important source of bias. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01