Theoretical Framework for Retrospective Studies of the Effectiveness of SARS-CoV-2 Vaccines

Abstract

Observational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. Such studies are now underway amid the ongoing rollout of SARS-CoV-2 vaccines globally. Although traditional case-control and test-negative design studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here, we review the theoretical basis for estimation of vaccine direct effects under traditional case-control and test-negative design frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs. Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, nonspecific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The traditional case-control design may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The test-negative design reduces but may not eliminate this confounding, for instance, if individuals who receive vaccination seek care or testing for less-severe illness. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources. We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages.

FIGURE.

Estimated vaccine effectiveness under the traditional case-control design and test-negative design with differential exposure to infection and prevalence of naturally acquired immunity among vaccinated and unvaccinated persons. We illustrate expected estimates of the odds ratio of vaccination given case versus control (or test-negative) status, under the limiting case of and , under scenarios where vaccination occurs among individuals with higher (left panel: , solid lines; , dashed lines) or lower (center panel: , solid lines; , dashed lines) risk of exposure to SARS-CoV-2. Colors correspond to differing proportions of the unvaccinated population remaining susceptible to infection (; we define the proportion of the vaccinated population remaining uninfected as for given values of . Red diagonal lines across the left and center panels illustrate the true effect. The right panel illustrates threshold values of , at which the effects of differential exposure to SARS-CoV-2 and naturally acquired immunity among vaccinated versus unvaccinated persons cancel out, resulting in a change in the direction of bias.