How to compare instrumental variable and conventional regression analyses using negative controls and bias plots

Abstract

There is increasing interest in the use of instrumental variable analysis to overcome unmeasured confounding in observational pharmacoepidemiological studies. This is partly because instrumental variable analyses are potentially less biased than conventional regression analyses. However, instrumental variable analyses are less precise, and regulators and clinicians find it difficult to interpret conflicting evidence from instrumental variable compared with conventional regression analyses. In this paper, we describe three techniques to assess which approach (instrumental variable versus conventional regression analyses) is least biased. These techniques are negative control outcomes, negative control populations and tests of covariate balance. We illustrate these methods using an analysis of the effects of smoking cessation therapies (varenicline) prescribed in primary care.

Keywords: Instrumental variables; causal inference; negative controls; pharmacoepidemiology.

Figure 1

Directed acyclic graph of outcome Y, prescription X, the instrumental variable Z and a potentially unmeasured confounder C (left). Each variable’s directed effects (edges) are denoted by arrows.

Figure 2

Directed acyclic graph of an analysis using the physicians’ prescriptions to their previous patients, Z* as a proxy for their preferences, the true underlying instrument, Z, which is a latent variable. The exposure, outcome and confounder are indicated as X, Y and C, respectively.

Figure 3

Proposed negative control outcomes and negative control populations.

Figure 4

Using urinary tract infections as a negative control outcome to investigate the effects of prescribing varenicline.

Figure 5

Bias component plots (left), are not informative without confidence intervals (right). Simulated bias component terms for 10 potential confounders (indicated c1 to c10) for the actual prescription (▪) and proposed instrument (). Simulation of 10 potential confounders when the instrument is valid. Using bias component plots alone we would erroneously conclude that the instrumental variable bias components were systematically larger than the linear regression bias components. Once we add confidence intervals to the point estimates, it becomes clear that the differences in components are entirely consistent with chance. There is no evidence from these potential confounders that the linear and instrumental variable regression bias component differ.

Figure 6

Negative control outcome: difference in the incidence of urinary tract infections in the four years after smoking cessation treatment for the index patients by actual prescription (▪) and the proposed instrument (). Horizontal lines indicate robust confidence intervals for each prescription. There is little evidence of differences in the prescribing history when the confidence intervals span zero on the axis.

Figure 7

Bias component plots: difference in patient’s age and the number of consultations in the previous year by actual exposure (▪) and proposed instrument (). The figures for the instrumental variable results account for the strength of the instrument as described in Jackson and Swanson (2015). The horizontal lines indicate robust confidence intervals for each prescription. There is little evidence of differences in the prescribing history when the confidence intervals span zero on the axis.

Figure 8

Bias component plots: difference in patients’ diagnoses in the previous year by actual exposure (▪) and proposed instrument (). The figures for the instrumental variable results account for the strength of the instrument as described in Jackson and Swanson (2015). The horizontal lines indicate robust confidence intervals for each prescription. There is little evidence of differences in the prescribing history when the confidence intervals span zero on the axis.

Figure 9

Bias component plots: difference in patients’ prescriptions received in the previous year by actual prescription (▪) and proposed instrument (). The figures for the instrumental variable results account for the strength of the instrument as described in Jackson and Swanson (2015). The horizontal lines indicate robust confidence intervals for each prescription. There is little evidence of differences in the prescribing history when the confidence intervals span zero on the axis.