International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality

Abstract

Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.

Fig. 1. Comparison of National Hospitalization Rates by Data Source.

Adjusted 7-day average new hospitalization rate and rate of ever-severe disease per 100,000 people by country based on 4CE contributors along with 95% confidence intervals compared with 7-day average new case rates collected by Johns Hopkins Center for Systems Science and Engineering (JHU CSSE).

Fig. 2. Risk Model Performance Across Countries and Continents.

AUCs of cox regression models with nine common laboratory tests (missing rate <30%) in predicting death adjusting for demographic variables and Charlson comorbidity index.

Fig. 3. Transportability of the Mortality Prediction Model Across Sites and Countries.

Heatmap of transportability of the Cox regression model across different sites and countries. Each part of the figure represents performance when the model is trained at one site and evaluated at another.

Fig. 4. Schematic of the federated EHR-based study involving healthcare systems from three countries.

Each site generated three data tables (comma-separated files) containing patient level data: 1) local patient clinical course indicates which days the patient was in the hospital and when the patient died; 2) local patient observation includes first three-character ICD9/10 diagnosis code and laboratory tests, where laboratory test has a numerical value; 3) local patient summary contains demographic variables including age, sex and race. Sites then conduct analysis using these individual level data within their firewall (see Methods).