Understanding COVID-19 Vaccine Effectiveness against Death Using a Novel Measure: COVID Excess Mortality Percentage

Abstract

COVID-19 vaccines have saved millions of lives; however, understanding the long-term effectiveness of these vaccines is imperative to developing recommendations for booster doses and other precautions. Comparisons of mortality rates between more and less vaccinated groups may be misleading due to selection bias, as these groups may differ in underlying health status. We studied all adult deaths during the period of 1 April 2021-30 June 2022 in Milwaukee County, Wisconsin, linked to vaccination records, and we used mortality from other natural causes to proxy for underlying health. We report relative COVID-19 mortality risk (RMR) for those vaccinated with two and three doses versus the unvaccinated, using a novel outcome measure that controls for selection effects. This measure, COVID Excess Mortality Percentage (CEMP), uses the non-COVID natural mortality rate (Non-COVID-NMR) as a measure of population risk of COVID mortality without vaccination. We validate this measure during the pre-vaccine period (Pearson correlation coefficient = 0.97) and demonstrate that selection effects are large, with non-COVID-NMRs for two-dose vaccinees often less than half those for the unvaccinated, and non-COVID NMRs often still lower for three-dose (booster) recipients. Progressive waning of two-dose effectiveness is observed, with an RMR of 10.6% for two-dose vaccinees aged 60+ versus the unvaccinated during April-June 2021, rising steadily to 36.2% during the Omicron period (January-June, 2022). A booster dose reduced RMR to 9.5% and 10.8% for ages 60+ during the two periods when boosters were available (October-December, 2021; January-June, 2022). Boosters thus provide important additional protection against mortality.

Keywords: COVID Excess Mortality Percentage; COVID-19; COVID-19 mortality; cause of death; selection bias; vaccine effectiveness.

Figure 1

Out-of-Sample Correlation for Indiana between Natural Mortality in 2019 and COVID-19 Mortality in 2020. Figure shows scatterplot of natural mortality in Indiana over April–December 2019 against COVID-19 mortality over April–December 2020, for groups defined by age (18–39, 40–49, 50–59, 60–69, 70–79, 80–89, 90+), gender, race/ethnicity, and zip-SES quintile. This figure also shows a best-fit regression line and a Pearson correlation coefficient. See Figure S2 for similar in-sample scatterplots for Wisconsin and Milwaukee County.

Figure 2

Actual versus Predicted Non-COVID Natural Mortality in Wisconsin. Figure shows monthly data for natural non-COVID and all natural deaths, for Wisconsin for January 2017–June 2022. For the pandemic period starting March 2020, we show both actual and predicted natural non-COVID deaths. Predicted deaths are based on linear extrapolation from 2017–2019 to the same calendar month during the pandemic period. Natural deaths (including COVID-19 deaths) are shown as solid red line; this shows two large COVID-related peaks in late 2020 and late 2021–early 2022. Natural non-COVID deaths (all natural deaths minus COVID-19 deaths) are shown as solid blue line. Predicted natural non-COVID deaths are shown as dashed green line.

Figure 3

RMR for Two-Dose and Three-Dose Elderly Vaccinees. Figure shows two-dose and three-dose relative mortality risk (RMR) versus the unvaccinated, by time period, for persons aged 60–79 and aged 80+.