Development of a machine learning algorithm to predict intubation among hospitalized patients with COVID-19

Abstract

Purpose: The purpose of this study is to develop a machine learning algorithm to predict future intubation among patients diagnosed or suspected with COVID-19.

Materials and methods: This is a retrospective cohort study of patients diagnosed or under investigation for COVID-19. A machine learning algorithm was trained to predict future presence of intubation based on prior vitals, laboratory, and demographic data. Model performance was compared to ROX index, a validated prognostic tool for prediction of mechanical ventilation.

Results: 4087 patients admitted to five hospitals between February 2020 and April 2020 were included. 11.03% of patients were intubated. The machine learning model outperformed the ROX-index, demonstrating an area under the receiver characteristic curve (AUC) of 0.84 and 0.64, and area under the precision-recall curve (AUPRC) of 0.30 and 0.13, respectively. In the Kaplan-Meier analysis, patients alerted by the model were more likely to require intubation during their admission (p < 0.0001).

Conclusion: In patients diagnosed or under investigation for COVID-19, machine learning can be used to predict future risk of intubation based on clinical data which are routinely collected and available in clinical setting. Such an approach may facilitate identification of high-risk patients to assist in clinical care.

Keywords: COVID-19; Intubation; Machine learning; Prediction; Respiratory distress.

Fig. 1

Time course of a subset of laboratory and vitals data for a patient from time of admission. Example data displayed include partial pressure of arterial oxygen (PAO2), partial pressure of arterial CO2 (PACO2), oxygen saturation (O2 SAT), Creatinine, Platelet count, white blood cell count (WBC), Temperature, Pulse, respiratory rate (RR), systolic blood pressure (Systolic), and Diastolic blood pressure (diastolic). Model predicted risk (blue, increased risk with increasing value), and ROX-index (green, increased risk with decreasing value), are displayed. Red line indicates the time of intubation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

Receiver operating characteristic (ROC) and precision-recall curves for the machine learning model (blue) and the ROX score (green). Dashed line for the ROC curve indicates random chance. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Kaplan-Meier plot showing the cumulative probability of remaining free of intubation from time of admission among patients alerted and not-alerted by the model. P-value indicates the result of a log-rank test.

Fig. 4

(A) Performance metrics AUC, AUPR, and average precision across various cohorts from the hold-out dataset. (B) Performance metrics stratified by sex, BMI ≥ 35, and hospital location.

Supplemental Fig. 1

Schematic of model framework. Using a sampling window of 24-h of vitals, laboratory, and demographics data, the model predicts presence of intubation 72 h from the end of the sampling window, designated as the forecast window. Predictions are made by sliding the sampling window every 12-h, and forecasting predictions 72-h from the end of the respective sampling window.

Supplemental Fig. 2

(A) Mean value of the decision criteria for time-series data used by the random forest model for forecasting predictions across all decision trees ± one standard deviation. (B) Rank-order of time-series and static variables feature importance (standardized score), with greater values indicating increased weight when making predictions by the model. Percent missing of each feature prior to forward-filling.