Evaluating the Impact of Meningococcal Vaccines With Synthetic Controls

Abstract

Invasive meningococcal disease (IMD) has a low and unpredictable incidence, presenting challenges for real-world evaluations of meningococcal vaccines. Traditionally, meningococcal vaccine impact is evaluated by predicting counterfactuals from pre-immunization IMD incidences, possibly controlling for IMD in unvaccinated age groups, but the selection of controls can influence results. We retrospectively applied a synthetic control (SC) method, previously used for pneumococcal disease, to data from 2 programs for immunization of infants against serogroups B and C IMD in England and Brazil. Time series of infectious/noninfectious diseases in infants and IMD cases in older unvaccinated age groups were used as candidate controls, automatically combined in a SC through Bayesian variable selection. SC closely predicted IMD in absence of vaccination, adjusting for nontrivial changes in IMD incidence. Vaccine impact estimates were in line with previous assessments. IMD cases in unvaccinated age groups were the most frequent SC-selected controls. Similar results were obtained when excluding IMD from control sets and using other diseases only, particularly respiratory diseases and measles. Using non-IMD controls may be important where there are herd immunity effects. SC is a robust and flexible method that addresses uncertainty introduced when equally plausible controls exhibit different post-immunization behaviors, allowing objective comparisons of IMD programs between countries.

Keywords: effectiveness; interrupted time series analysis; invasive meningococcal disease; meningococcal infections; synthetic controls; vaccine impact; vaccines.

Figure 1

Vaccine impact estimates in non-vaccine-eligible age groups: 5–9 years (A and D), 10–14 years (B and E), and 15–19 years(C and F) in Brazil (A, B, and C), 2007–2013, and England (panels D, E, and F), 2011–2019, using different models (synthetic control 1 and 2 (SC1 and SC2) shown as circles; interrupted time series (ITS) and controlled ITS (CITS) as diamonds). CITS-L, CITS with all meningococcal serogroup B (MenB) (England)/meningococcal serogroup C (MenC) (Brazil) cases in non-vaccine-eligible age groups used as controls (excluding the target) and incorporating changes in level only; CITS-S, same as CITS-L, but incorporating changes in both level and slope; ITS-L, interrupted time series incorporating changes in level only; ITS-S, ITS incorporating changes in both level and slope; SC1, synthetic control method using all the controls available; SC2, synthetic control method excluding IMD cases in non-vaccine-eligible from the set of candidate controls.

Figure 2

Meningococcal cases predicted by the synthetic control 1 and 2 (SC1 and SC2) models for meningococcal serogroup C (MenC) (Brazil, 2007–2013) (A, B, and C) and meningococcal serogroup B (MenB) (England, 2011–2019) (D, E, and F) disease among nonvaccinated persons in the age groups 5–9(A and D), 10–14 (B and E), and 15–19 (C and F) years. In dark blue, cases predicted with the SC method using all the controls available (SC1) (curve: best estimate; shaded region: 95% credible interval (CrI)). In light blue, cases predicted excluding MenB/MenC cases in unvaccinated age groups (SC2) (curve: best estimate; shaded region: 95% CrI). Observed data reported as black dots. The model was fitted on prevaccination data only (best fits shown as solid lines). Postintervention predictions (i.e., counterfactuals) shown as dashed lines.

Figure 3

Meningococcal cases predicted by the synthetic control 1 (SC1) model for meningococcal serogroup C (MenC) disease (Brazil, 2007–2013) in vaccine-eligible age groups of <1-year-olds (A) and 1- to 4-year-olds (B) and meningococcal serogroup B (MenB) disease (England, 2011–2019) in 18- to 51-week-olds (C) and 1-year-olds (D). In black, cases predicted with the SC method (curve: best estimate; shaded region: 95% credible interval (CrI)). Observed cases are shown as black dots. Solid black vertical lines indicate the introduction of the vaccination campaign. Dashed gray vertical lines indicate the initial point for measuring impact.

Figure 4

Top 3 selected controls with highest probability of inclusion, for the <1-year-olds age group and 1- to 4-year-olds age group in Brazil(A and B, respectively), 2007–2013; and for the 18- to 51-week-olds age group and 1-year-olds age group in England (C and D, respectively), 2011–2019. We report results using all the controls (black bars) and a subset where meningococcal serogroup B (MenB)/ meningococcal serogroup C (MenC) cases in non-vaccine-eligible age groups were excluded (white bars). ACH, aggregated variable with all the controls summed together; D50–89, diseases of blood and blood-forming organs and certain disorders involving the immune mechanism; E00–99, endocrine, nutritional, metabolic disorders; E40–46, malnutrition; I00–99, diseases of the circulatory system; J20–J22, bronchitis, bronchiolitis, and unspecified acute lower respiratory infection; NE, noneligible age group; P00–99, perinatal diseases; RSV, respiratory syncytial virus; S00–T98, injury, poisoning, and consequences of external causes.

Figure 5

Vaccine impact estimates for meningococcal serogroup C (MenC) (Brazil, 2007–2013) disease in the vaccine-eligible age groups of <1- and 1- to 4-year-olds (A) and meningococcal serogroup B (MenB) (England, 2011–2019) disease in the 18- to 51-week-olds and 1-year-olds (B) when using the synthetic control 1 and 2 (SC1 and SC2) models (black and white dots). 95% credible intervals (CrIs) are shown as gray lines. Vaccine impact estimates using all the controls available are shown as black dots. Vaccine impact estimates excluding MenB/MenC cases in unvaccinated age groups are shown as white dots. NE, noneligible age group.