Flaws and uncertainties in pandemic global excess death calculations

Abstract

Several teams have been publishing global estimates of excess deaths during the COVID-19 pandemic. Here, we examine potential flaws and underappreciated sources of uncertainty in global excess death calculations. Adjusting for changing population age structure is essential. Otherwise, excess deaths are markedly overestimated in countries with increasingly aging populations. Adjusting for changes in other high-risk indicators, such as residence in long-term facilities, may also make a difference. Death registration is highly incomplete in most countries; completeness corrections should allow for substantial uncertainty and consider that completeness may have changed during pandemic years. Excess death estimates have high sensitivity to modelling choice. Therefore different options should be considered and the full range of results should be shown for different choices of pre-pandemic reference periods and imposed models. Any post-modelling corrections in specific countries should be guided by pre-specified rules. Modelling of all-cause mortality (ACM) in countries that have ACM data and extrapolating these models to other countries is precarious; models may lack transportability. Existing global excess death estimates underestimate the overall uncertainty that is multiplicative across diverse sources of uncertainty. Informative excess death estimates require risk stratification, including age groups and ethnic/racial strata. Data to-date suggest a death deficit among children during the pandemic and marked socioeconomic differences in deaths, widening inequalities. Finally, causal explanations require great caution in disentangling SARS-CoV-2 deaths, indirect pandemic effects and effects from measures taken. We conclude that excess deaths have many uncertainties, but globally deaths from SARS-CoV-2 may be the minority of calculated excess deaths.

Keywords: COVID-19; bias; death certificates; demography; excess deaths; mortality.

Figure 1.

Cumulative recorded COVID-19 deaths, excess death estimates according to the Economist, and excess death estimates with age-adjustment (using 2017-1019 as reference) for the 3 years’ period 2020-2022 in 4 countries with reliable death registration systems. See references (3) and (4) for the methods underlying the excess death calculations. Data are presented as deaths per 100 people in the general population. For Sweden, the age-adjusted excess death calculation shows a death deficit during 2020-2022 (fewer deaths than 2017-2019 after age-adjustment). The total deaths (from all causes) in 2020-2022 were as follows: 3.1% of the population died in the USA, 3.7% in Germany, 3.0% in Netherlands, 2.7% in Sweden.

Figure 2.

Cumulative recorded COVID-19 deaths, excess death estimates according to the Economist, and expected COVID-19 deaths for the period from January 2020 to end of March 2023 in 4 countries without reliable death registration systems. See reference (3) for the methods underlying the excess death calculations. The expected COVID-19 deaths are a rough, speculative estimate; their calculation assumes that almost everyone has been infected and the overall population infection fatality rate was 0.1% for India and Indonesia, 0.05% for Kenya, and 0.02% for China. See Appendix for illustrative calculations of infection fatality rate and other considerations in India. The population of Indonesia has a similar age structure to India. Median age (proportion of population over 65 years) is 28 years (6.8%) in India versus 29 years (6.9%) in Indonesia. Median age is 19 years (2.8% above 65 years of age) in Kenya thus population-level infection fatality rate was probably half (or less) compared with India/Indonesia). In these 3 countries, probably most people were infected before any vaccination, thus age-stratified infection fatality rates for pre-vaccination era (50,51) may apply to most people (IFR may be lower for those vaccinated before being infected). For China, see reference for plausible estimates of COVID-19 deaths during the massive Omicron wave following removal of zero COVID policy measures All mortality estimates in these 4 countries carry large uncertainty.