Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness

Abstract

Influenza viruses undergo frequent antigenic changes. As a result, the viruses circulating change within and between seasons, and the composition of the influenza vaccine is updated annually. Thus, estimation of the vaccine's effectiveness is not constant across seasons. In order to provide annual estimates of the influenza vaccine's effectiveness, health departments have increasingly adopted the "test-negative design," using enhanced data from routine surveillance systems. In this design, patients presenting to participating general practitioners with influenza-like illness are swabbed for laboratory testing; those testing positive for influenza virus are defined as cases, and those testing negative form the comparison group. Data on patients' vaccination histories and confounder profiles are also collected. Vaccine effectiveness is estimated from the odds ratio comparing the odds of testing positive for influenza among vaccinated patients and unvaccinated patients, adjusting for confounders. The test-negative design is purported to reduce bias associated with confounding by health-care-seeking behavior and misclassification of cases. In this paper, we use directed acyclic graphs to characterize potential biases in studies of influenza vaccine effectiveness using the test-negative design. We show how studies using this design can avoid or minimize bias and where bias may be introduced with particular study design variations.

Keywords: causal inference; directed acyclic graphs; epidemiologic methods; influenza; observational studies; test-negative study design; vaccine effectiveness.

Figure 1.

Simple directed acyclic graph of influenza vaccine effectiveness with confounding. The causal effect of interest is the effect of vaccination V on influenza I, shown by the arrow from V to I. A) Potential confounders include age A, high-risk status HR, calendar time CT, and sex S. For each potential confounder, there is a directed path to both I and V. B) The rectangles around each confounder indicate adjustment in a statistical model and closure of the biasing paths (directed paths are removed).

Figure 2.

Directed acyclic graph illustrating the time-varying effects of prior exposure to the influenza vaccine and influenza virus across 2 seasons. The current season is depicted with a subscript 2, while the prior season is depicted with a subscript 1. The effect of interest is that of vaccination status in the current season V₂ on current season influenza status I₂. Influenza status in the previous season may influence a person's decision to be vaccinated in the current season (I₁→V₂) and will affect his/her susceptibility to infection in the current season, if the strains are similar or confer some cross-protection (I₁→I₂). Vaccination in the previous season influences vaccination in the current season (V₁→V₂), and protection may linger until the current season (V₁→I₂). Confounder status in the previous season C₁, particularly age and high-risk status, is an ancestor of, but may differ from, confounder status in the current season C₂ (C₁→C₂) and will influence both vaccination status (C₁→V₁) and influenza status (C₁→I₁).

Figure 3.

Directed acyclic graph illustrating confounding by health-care-seeking behavior. A) Health-care-seeking HS confounds the relationship between vaccination V and influenza infection I. The parentheses indicate that HS is unmeasured. Additional confounders C may be parents of HS (C→HS). B) By including only patients with positive health-care-seeking behaviors (shown by the rectangle around HS = 1), the test-negative design implicitly removes this source of confounding (biasing paths removed).

Figure 4.

Directed acyclic graph illustrating selection bias by health-care-seeking behavior. A) Health-care-seeking HS confounds the relationship between vaccination V and influenza status I and also influences testing and selection into the study T. Influenza status may also influence a person to seek care and be tested. Additional confounders of the V→I relationship are represented by C and may also influence HS. Only patients who are tested for influenza (T = 1) are included in the study, resulting in collider bias. B) Control of HS (HS = 1) blocks the biasing path.

Figure 5.

Directed acyclic graph illustrating misclassification of outcome status in a test-negative study. I* indicates measured influenza status. A) U_I includes all of the factors that influence measurement of influenza status (U_I →I*). B) U_I is separated into known components, including availability of a test result T, viral shedding VS, and sample quality SQ, as well as other components that remain unknown U_I.

Figure 6.

Directed acyclic graph illustrating differential misclassification of influenza status. Influenza status I may be differentially ascertained I* with respect to vaccination status V due to reduced viral shedding VS among recipients of live attenuated influenza vaccine.

Figure 7.

Directed acyclic graph illustrating independent, nondifferential misclassification of both influenza and vaccination status. True influenza status I may be misclassified I* due to the effects of factors which contribute to the measurement of influenza status U_I. True vaccination status V may be misclassified V* due to the effects of factors which contribute to the measurement of vaccination status U_V. C indicates other confounders of V→I and in this graph are assumed to be independent of I* and V*.

Figure 8.

Directed acyclic graph illustrating misclassification of influenza and vaccination status within levels of a confounder. Age A affects detection of influenza I* because the duration of viral shedding VS differs by age group (A→VS), and viral shedding is needed to detect influenza (VS→I*), leading to misclassification of influenza status that is dependent on age (A→VS→I*). Recall R of vaccination status V may be poorer among the elderly (A→R), which in turn influences measured vaccination status (R→V*), leading to misclassification of vaccination status that is dependent on age (A→R→V).