Inferring the effectiveness of government interventions against COVID-19

Abstract

Governments are attempting to control the COVID-19 pandemic with nonpharmaceutical interventions (NPIs). However, the effectiveness of different NPIs at reducing transmission is poorly understood. We gathered chronological data on the implementation of NPIs for several European and non-European countries between January and the end of May 2020. We estimated the effectiveness of these NPIs, which range from limiting gathering sizes and closing businesses or educational institutions to stay-at-home orders. To do so, we used a Bayesian hierarchical model that links NPI implementation dates to national case and death counts and supported the results with extensive empirical validation. Closing all educational institutions, limiting gatherings to 10 people or less, and closing face-to-face businesses each reduced transmission considerably. The additional effect of stay-at-home orders was comparatively small.

Median intervention effectiveness estimates across a suite of 206 analyses with different epidemiological parameters, data, and modeling assumptions.

Bayesian inference using a semimechanistic hierarchical model with observed national case and death data across 41 countries between January and May 2020 is used to infer the effectiveness of several nonpharmaceutical interventions. Although precise effectiveness estimates depend on the assumed data and parameters, there are clear trends across the experimental conditions. Violins show kernel density estimates of the posterior median effectiveness across the sensitivity analysis. R_t, instantaneous reproduction number.

Fig. 1. Timing of NPI implementations in early 2020.

Crossed-out icons signify when an NPI was lifted. Detailed definitions of the NPIs are given in Table 1.

Fig. 2. NPI effectiveness under default model settings.

Posterior percentage reductions in R_t with median, 50%, and 95% prediction intervals shown. Prediction intervals reflect many sources of uncertainty, including NPI effectiveness varying by country and uncertainty in epidemiological parameters. A negative 1% reduction refers to a 1% increase in R_t. “Schools and universities closed” shows the joint effect of closing both schools and universities; the individual effect of closing just one will be smaller (see text). Cumulative effects are shown for hierarchical NPIs (gathering bans and business closures), that is, the result for “Most nonessential businesses closed” shows the cumulative effect of two NPIs with separate parameters and icons—closing some (high-risk) businesses, and additionally closing most remaining (non-high-risk but nonessential) businesses given that some businesses are already closed.

Fig. 3. Combined NPI effectiveness for the 15 most commonly implemented sets of NPIs in our data.

Black and gray bars denote 50% and 95% Bayesian prediction intervals, respectively. (A) Predicted R_t after implementation of each set of NPIs, assuming R₀ = 3.3. (B) Maximum R₀ that can be reduced to R_t below 1 by common sets of NPIs. Readers can interactively explore the effects of all sets of NPIs, while setting R₀ and adjusting NPI effectiveness to local circumstances, with our online mitigation calculator (16).

Fig. 4. Median NPI effectiveness across the sensitivity analyses.

(A) Median NPI effectiveness (reduction in R_t) when varying different components of the model or the data in 206 experimental conditions. Results are displayed as violin plots, using kernel density estimation to create the distributions. Inside the violins, the box plots show median and interquartile range. The vertical lines mark 0, 17.5, and 35% (see text). (B to E) Categorized sensitivity analyses. (B) Sensitivity to model structure. Using only cases or only deaths as observations (two experimental conditions; fig. S7); varying the model structure (three conditions; fig. S8, left). (C) Sensitivity to data and preprocessing. Leaving out countries from the dataset (42 conditions; figs. S5 and S21); varying the threshold below which cases and deaths are masked (eight conditions; fig. S13); sensitivity to correcting for undocumented cases and to country-level differences in case ascertainment (two conditions; fig. S6). (D) Sensitivity to epidemiological parameters. Jointly varying the means of the priors over the means of the generation interval, the infection-to-case-confirmation delay, and the infection-to-death delay (125 conditions; fig. S10); varying the prior over R₀ (four conditions; fig. S11); varying the prior over NPI effect parameters (three conditions; fig. S11); varying the prior over the degree to which NPI effects vary across countries (three conditions; fig. S12). (E) Sensitivity to unobserved factors influencing R_t. Excluding observed NPIs one at a time (eight conditions; fig. S9); controlling for additional NPIs from a different dataset (six conditions; fig. S9).

Fig. 5. Model overview.

Unshaded, white nodes are observed. From bottom to top: The mean effect parameter of NPI i is α_i, and the country-specific effect parameter is α_i,c. On each day t, a country’s daily reproduction number R_t,c depends on the country’s basic reproduction number R_0,c and the active NPIs. The active NPIs are encoded by x_i,t,c, which is 1 if NPI i is active in country c at time t, and 0 otherwise. R_t,c is transformed into the daily growth rate g_t,c using the generation interval parameters and subsequently is used to compute the new infections $N_{t,c}^{(\mathrm{C})}$ and $N_{t,c}^{(\mathrm{D})}$ that will subsequently become confirmed cases and deaths, respectively. Finally, the expected numbers of daily confirmed cases $y_{t,c}^{(\mathrm{C})}$ and deaths $y_{t,c}^{(\mathrm{D})}$ are computed using discrete convolutions of $N_{t,c}^{(.)}$ with the relevant delay distributions. Our model uses both case and death data; it splits all nodes above the daily growth rate g_t,c into separate branches for deaths and confirmed cases. We account for uncertainty in the generation interval, infection-to–case confirmation delay, and the infection-to-death delay by placing priors over the parameters of these distributions.