A predictive model of clinical deterioration among hospitalized COVID-19 patients by harnessing hospital course trajectories

Abstract

From early March through mid-May 2020, the COVID-19 pandemic overwhelmed hospitals in New York City. In anticipation of ventilator shortages and limited ICU bed capacity, hospital operations prioritized the development of prognostic tools to predict clinical deterioration. However, early experience from frontline physicians observed that some patients developed unanticipated deterioration after having relatively stable periods, attesting to the uncertainty of clinical trajectories among hospitalized patients with COVID-19. Prediction tools that incorporate clinical variables at one time-point, usually on hospital presentation, are suboptimal for patients with dynamic changes and evolving clinical trajectories. Therefore, our study team developed a machine-learning algorithm to predict clinical deterioration among hospitalized COVID-19 patients by extracting clinically meaningful features from complex longitudinal laboratory and vital sign values during the early period of hospitalization with an emphasis on informative missing-ness. To incorporate the evolution of the disease and clinical practice over the course of the pandemic, we utilized a time-dependent cross-validation strategy for model development. Finally, we validated our prediction model on an external validation cohort of COVID-19 patients served in a demographically distinct population from the training cohort. The main finding of our study is the identification of risk profiles of early, late and no clinical deterioration during the course of hospitalization. While risk prediction models that include simple predictors at ED presentation and clinical judgement are able to identify any deterioration vs. no deterioration, our methodology is able to isolate a particular risk group that remain stable initially but deteriorate at a later stage of the course of hospitalization. We demonstrate the superior predictive performance with the utilization of laboratory and vital sign data during the early period of hospitalization compared to the utilization of data at presentation alone. Our results will allow efficient hospital resource allocation and will motivate research in understanding the late deterioration risk group.

Keywords: COVID-19; Deterioration; EMR; Intubation; Machine learning; Prediction.

Graphical abstract

Fig. 1

Time-dependent cross-validation scheme.

Fig. 2

Examples of LGMM clusters of trajectories. FiO2 (left), level of supplemental oxygen (center), respiratory rate (right) Red is for Cluster 1 and blue is for Cluster 2. Solid line: locally weighted scatterplot smoothing (LOESS) curve Shades: 95% confidence interval.

Fig. 3

Predictor importance from RSF on training cohort (NYP-WCM). The top most important predictors determined as those that explain 70% of the total cumulative importance are shown. ‘age_gt65’=≥65 years of age. For labs and vitals, predictors are labeled as ‘_’. Features are labeled as below: ‘cluster_1_9’=Low/decreasing (Cluster 1) compared to high/increasing (Cluster 2) or no data/missing value (Cluster 9). Clusters were identified using LGMM. ‘cluster_2_9’=High/increasing (Cluster 2) compared to low/decreasing (Cluster 1) or no data/missing value (Cluster 9). Clusters were identified using LGMM. ‘nodata’=No available data or missing lab or vital sign ‘n_per_day_trend’=Trend in the number of values recorded per calendar day Labs and vital signs are labeled as below: ‘supp_oxygen’=Level of supplemental oxygen. ‘resp_rate’=Respiratory rate.

Fig. 4

Kaplan-Meier estimates by risk profile: training cohort (NYP-WCM).

Fig. 5

Kaplan-Meier estimates by risk profile: validation Cohort (NYP-LMH).