ASSESSING SELECTION BIAS IN REGRESSION COEFFICIENTS ESTIMATED FROM NONPROBABILITY SAMPLES WITH APPLICATIONS TO GENETICS AND DEMOGRAPHIC SURVEYS

Abstract

Selection bias is a serious potential problem for inference about relationships of scientific interest based on samples without well-defined probability sampling mechanisms. Motivated by the potential for selection bias in: (a) estimated relationships of polygenic scores (PGSs) with phenotypes in genetic studies of volunteers and (b) estimated differences in subgroup means in surveys of smartphone users, we derive novel measures of selection bias for estimates of the coefficients in linear and probit regression models fitted to nonprobability samples, when aggregate-level auxiliary data are available for the selected sample and the target population. The measures arise from normal pattern-mixture models that allow analysts to examine the sensitivity of their inferences to assumptions about nonignorable selection in these samples. We examine the effectiveness of the proposed measures in a simulation study and then use them to quantify the selection bias in: (a) estimated PGS-phenotype relationships in a large study of volunteers recruited via Facebook and (b) estimated subgroup differences in mean past-year employment duration in a nonprobability sample of low-educated smartphone users. We evaluate the performance of the measures in these applications using benchmark estimates from large probability samples.

Keywords: Linear regression; National Survey of Family Growth; nonprobability samples; polygenic scores; probit regression; selection bias.

Fig. 1.

Scatter plots presenting associations between MUBNS and the true differences in coefficients between selected and nonselected units for the Z₁ coefficient. Results are median MUBNS values across 1000 simulated datasets for each of the 1944 combinations of data generation model and selection mechanism; panels are separated by the level of dependence on Y in the selection model (OR_Y; rows) and the correlation between Y and A, given Z₁ and Z₂ (columns). The dotted black line represents the Y = X relationship.

Fig. 2.

Side-by-side box plots presenting distributions of the Spearman correlations between MUBNS and the true difference in the coefficients between selected and nonselected units. We estimate each correlation from 1000 replicate populations for each combination of data generation model and selection model. OR_A = odds ratio for A in the selection model; OR_Y = odds ratio for Y in the selection model.

Fig. 3.

Side-by-side box plots presenting distributions of the empirical coverage rates for the alternative intervals. We estimate each coverage rate by computing the interval for each coefficient from 1000 replicate populations for each combination of data generation model and selection model. OR_A = odds ratio for A in the selection model; OR_Y = odds ratio for Y in the selection model. The horizontal black line represents 0.95 coverage, for reference.

Fig. 4.

Side-by-side box plots presenting distributions of the empirical median widths for the alternative intervals across the different scenarios. We obtain the median width by computing the interval for each coefficient from 1000 replicate populations for each combination of data generation model and selection model. OR_A = odds ratio for A in the selection model; OR_Y = odds ratio for Y in the selection model.