Monte Carlo Simulation Approaches for Quantitative Bias Analysis: A Tutorial

Abstract

Quantitative bias analysis can be used to empirically assess how far study estimates are from the truth (i.e., an estimate that is free of bias). These methods can be used to explore the potential impact of confounding bias, selection bias (collider stratification bias), and information bias. Quantitative bias analysis includes methods that can be used to check the robustness of study findings to multiple types of bias and methods that use simulation studies to generate data and understand the hypothetical impact of specific types of bias in a simulated data set. In this article, we review 2 strategies for quantitative bias analysis: 1) traditional probabilistic quantitative bias analysis and 2) quantitative bias analysis with generated data. An important difference between the 2 strategies relates to the type of data (real vs. generated data) used in the analysis. Monte Carlo simulations are used in both approaches, but the simulation process is used for different purposes in each. For both approaches, we outline and describe the steps required to carry out the quantitative bias analysis and also present a bias-analysis tutorial demonstrating how both approaches can be applied in the context of an analysis for selection bias. Our goal is to highlight the utility of quantitative bias analysis for practicing epidemiologists and increase the use of these methods in the epidemiologic literature.

Keywords: Monte Carlo sampling; bias analysis; confounding; measurement error; misclassification; selection bias; simulation study.

Figure 1

Directed acyclic graphs showing A) confounding bias, B) selection bias (collider stratification bias), C) information bias (nondifferential exposure misclassification). C, confounder; D, disease; E, exposure; E^*, exposure as measured; L, common cause of collider (S) and disease; S, selection into sample; U_E, sources of error in measurement of exposure as measured.

Figure 2

General framework for probabilistic quantitative bias analysis.

Figure 3

Calculating bias parameters for quantitative bias analysis of an unmeasured confounder. ϕ, effect of the confounder on exposure; θ, effect of the confounder on disease.

Figure 4

Framework for quantitative bias analysis from a simulated (user generated) data set with data generation under specific causal scenarios. DAG, directed acyclic graph.

Figure 5

Directed acyclic graph representing the causal structure for the relationship between osteoporosis and dementia.

Figure 6

Estimated odds ratios (ORs) and 95% confidence intervals from (A) 1,000 simulated samples in the total population (mean OR = 1.00) and (B) among those with hip fracture (mean OR = 0.66). Dashed line indicates null value (OR = 1.00).