Understanding COVID-19 Vaccine Effectiveness Against Death Using a Novel Measure: COVID Excess Mortality Percentage

Abstract

COVID-19 vaccines have saved millions of lives and prevented countless adverse patient disease outcomes. Understanding the long-term effectiveness of these vaccines is imperative to developing recommendations for precautions and booster doses. Comparisons between more and less vaccinated groups may be misleading due to selection bias, as these groups may differ in underlying health status and thus risk of adverse COVID-19 outcomes. We study all adult deaths over April 1, 2021-June 30, 2022 in Milwaukee County, Wisconsin, linked to vaccination records, use mortality from other natural causes to proxy for underlying health, and report relative COVID-19 mortality risk (RMR) for vaccinees versus the unvaccinated, using a novel outcome measure that controls for selection effects. This measure, COVID Excess Mortality Percentage (CEMP) uses the non-COVID natural mortality rate (Non-Covid-NMR) as a measure of population risk of COVID mortality without vaccination. We validate this measure during the pre-vaccine period (r = 0.97) and demonstrate that selection effects are large, with Non-Covid-NMRs for two-dose vaccinees less than half those for the unvaccinated, and Non-COVID NMRs still lower for three dose (booster) recipients. Progressive waning of two-dose effectiveness is observed, with relative mortality risk (RMR) for two-dose vaccinees aged 60 + versus the unvaccinated of 11% during April-June 2021, rising steadily to 36% during the Omicron period (January-June, 2022). Notably, a booster dose reduced RMR to 10-11% for ages 60+. Boosters thus provide important additional protection against mortality.

Figure 1. Out-of-Sample Correlation for Indiana between Natural Mortality in 2019 and COVID-19 Mortality in 2020.

Figure shows scatterplot of natural mortality in Indiana over April-December 2019 against COVID-19 mortality over April-December 2020, for groups defined by age (18–39, 40–49, 50–59, 60–69, 70–79, 80–89, 90+)*gender*race/ethnicity*zip-SES quintile, best-fit regression line, and Pearson correlation coefficient. See Figure App-1 for similar in-sample scatter plots for Wisconsin and Milwaukee County

Figure 2. Actual versus Predicted Non-COVID Natural Mortality in Wisconsin

Figure shows monthly data for natural non-COVID-19 and all natural deaths, for Wisconsin for January 2017 – June 2022. For the pandemic period starting March 2020, we show both actual and predicted natural non-COVID deaths. Predicted deaths are based on linear extrapolation from 2017–2019 to the same calendar month during the pandemic period. Natural deaths (including COVID-19 deaths) are shown as solid red line; this shows two large COVID-related peaks in late 2020 and late-2021-early 2022. Natural non-COVID-19 deaths (all natural deaths minus COVID-19 deaths) are shown as solid blue line. Predicted natural non-COVID deaths are shown as dashed green line.

Figure 3. RMR for Two-Dose and Three-Dose Elderly Vaccinees

Figure shows two-dose and three-dose relative mortality risk (RMR) versus the unvaccinated, by time period, for persons aged 60–79 and aged 80+.