Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Sep;37(3):751-769.
doi: 10.2478/jos-2021-0033. Epub 2021 Sep 12.

A simulation study of diagnostics for selection bias

Philip S Boonstra  1 , Roderick J A Little  1   2 , Brady T West  2 , Rebecca R Andridge  3 , Fernanda Alvarado-Leiton  2
Affiliations

A simulation study of diagnostics for selection bias

Philip S Boonstra et al. J Off Stat. 2021 Sep.

Abstract

A non-probability sampling mechanism arising from non-response or non-selection is likely to bias estimates of parameters with respect to a target population of interest. This bias poses a unique challenge when selection is 'non-ignorable', i.e. dependent upon the unobserved outcome of interest, since it is then undetectable and thus cannot be ameliorated. We extend a simulation study by Nishimura et al. [International Statistical Review, 84, 43-62 (2016)], adding two recently published statistics: the so-called 'standardized measure of unadjusted bias (SMUB)' and 'standardized measure of adjusted bias (SMAB)', which explicitly quantify the extent of bias (in the case of SMUB) or non-ignorable bias (in the case of SMAB) under the assumption that a specified amount of non-ignorable selection exists. Our findings suggest that this new sensitivity diagnostic is more correlated with, and more predictive of, the true, unknown extent of selection bias than other diagnostics, even when the underlying assumed level of non-ignorability is incorrect.

Keywords: Multiple Imputation; Non-Ignorable Selection Bias; Pattern Mixture Model; Survey Non-Response.

PubMed Disclaimer

Conflict of interest statement

Disclosure The authors report no potential conflicts of interest.

Figures

Figure 1:
Figure 1:
Median standardized error measure (SEM, y-axes) against value of diagnostic (x-axes) for twelve candidate diagnostics (columns), three values of ρ ≡ Cor(X1, Y) (rows) using the median of 2000 simulated datasets. κ ≡ Cor(X1,X2) is fixed at 1 (Figures 2 and 3 give the same results for κ = 0:5 and κ = 0, respectively) For reference, the y = x line is plotted in black. Shape and color indicate different true selection mechanisms from Table 2, and connected segments represent different values of {βx, βy} corresponding to the same selection mechanism.
Figure 2:
Figure 2:
Median standardized error measure (SEM, y-axes) against value of diagnostic (x-axes) for ten candidate diagnostics (columns), two values of ρ ≡ Cor(X1, Y) (rows) using the median of 2000 simulated datasets. κ ≡ Cor(X1,X2) is fixed at 0.5 (Figures 1 and 3 give the same results for κ = 1 and κ = 0, respectively). For reference, the y = x line is plotted in black. Shape and color indicate different true selection mechanisms from Table 2, and connected segments represent different values of {βx, βy} corresponding to the same selection mechanism.
Figure 3:
Figure 3:
Median standardized error measure (SEM, y-axes) against value of diagnostic (x-axes) for ten candidate diagnostics (columns), two values of ρ ≡ Cor(X1, Y) (rows) using the median of 2000 simulated datasets. κ ≡ Cor(X1,X2) is fixed at 0 (Figures 1 and 2 give the same results for κ = 1 and κ = 0:5, respectively). For reference, the y = x line is plotted in black. Shape and color indicate different true selection mechanisms from Table 2, and connected segments represent different values of {βx, βy} corresponding to the same selection mechanism.
See this image and copyright information in PMC

References

    1. Albert A and Anderson J (1984). On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71, 1–10.
    1. Andridge RR and Little RJ (2011). Proxy pattern-mixture analysis for survey nonresponse. Journal of Official Statistics 27, 153–180.
    1. Andridge RR and Little RJ (2020). Proxy pattern-mixture analysis for a binary variable subject to nonresponse. Journal of Official Statistics To Appear,.
    1. Bootsma-van der Wiel A. v., Van Exel E, De Craen A, Gussekloo J, Lagaay A, Knook D, and Westendorp R (2002). A high response is not essential to prevent selection bias: results from the leiden 85-plus study. Journal of Clinical Epidemiology 55, 1119–1125. - PubMed
    1. Brick JM and Williams D (2013). Explaining rising nonresponse rates in cross-sectional surveys. The Annals of the American Academy of Political and Social Science 645, 36–59.

LinkOut - more resources