The Effects of Non-pharmaceutical Interventions on COVID-19 Mortality: A Generalized Synthetic Control Approach Across 169 Countries

Abstract

Importance: Governments have introduced non-pharmaceutical interventions (NPIs) in response to the pandemic outbreak of Coronavirus disease (COVID-19). While NPIs aim at preventing fatalities related to COVID-19, the previous literature on their efficacy has focused on infections and on data of the first half of 2020. Still, findings of early NPI studies may be subject to underreporting and missing timeliness of reporting of cases. Moreover, the low variation in treatment timing during the first wave makes identification of robust treatment effects difficult.

Objective: We enhance the literature on the effectiveness of NPIs with respect to the period, the number of countries, and the analytical approach.

Design setting and participants: To circumvent problems of reporting and treatment variation, we analyse data on daily confirmed COVID-19-related deaths per capita from Our World in Data, and on 10 different NPIs from the Oxford COVID-19 Government Response Tracker (OxCGRT) for 169 countries from 1st July 2020 to 1st September 2021. To identify the causal effects of introducing NPIs on COVID-19-related fatalities, we apply the generalized synthetic control (GSC) method to each NPI, while controlling for the remaining NPIs, weather conditions, vaccinations, and NPI-residualized COVID-19 cases. This mitigates the influence of selection into treatment and allows to model flexible post-treatment trajectories.

Results: We do not find substantial and consistent COVID-19-related fatality-reducing effects of any NPI under investigation. We see a tentative change in the trend of COVID-19-related deaths around 30 days after strict stay-at-home rules and to a slighter extent after workplace closings have been implemented. As a proof of concept, our model is able to identify a fatality-reducing effect of COVID-19 vaccinations. Furthermore, our results are robust with respect to various crucial sensitivity checks.

Conclusion: Our results demonstrate that many implemented NPIs may not have exerted a significant COVID-19-related fatality-reducing effect. However, NPIs might have contributed to mitigate COVID-19-related fatalities by preventing exponential growth in deaths. Moreover, vaccinations were effective in reducing COVID-19-related deaths.

Keywords: COVID-19; global public health; health policy; lockdown; non-pharmaceutical interventions; vaccination.

Figure 1

The effect of NPIs on COVID-19 deaths. Generalized synthetic control estimator based on daily data. Black solid lines represent the average treatment effects on the treated (ATTs). Ribbons represent 95% non-parametric confidence intervals based on 1,000 bootstrap runs. Dotted lines are the null lines. Dashed lines represent linear predictions based on the 35 days before the intervention. Controls: 9 remaining NPIs as stringency index, temperature, temperature², cloud cover, precipitation, humidity, total vaccinations, 7-day backwards rolling average of NPI-residualized COVID-19 cases at t – 7, t – 14, t – 21, t – 28, and t – 35.

Figure 2

The effect of vaccinations (vaccine doses per inhabitant ≥80 %) on COVID-19 deaths. Generalized synthetic control estimator based on daily data. The black solid line represents the average treatment effect on the treated (ATT). Ribbons represent 95% non-parametric confidence intervals based on 1,000 bootstrap runs. The dotted line is the null line. The dashed line represents the linear prediction based on the 35 days before the intervention. Controls: 10 NPIs as stringency index, temperature, temperature², cloud cover, precipitation, humidity, 7-day backwards rolling average of NPI-residualized COVID-19 cases at t – 7, t – 14, t – 21, t – 28, and t – 35.