Machine Learning-Based Identification of Risk Factors for ICU Mortality in 8902 Critically Ill Patients with Pandemic Viral Infection

Abstract

Background/Objectives: The SARS-CoV-2 and influenza A (H1N1)pdm09 pandemics have resulted in high numbers of ICU admissions, with high mortality. Identifying risk factors for ICU mortality at the time of admission can help optimize clinical decision making. However, the risk factors identified may differ, depending on the type of analysis used. Our aim is to compare the risk factors and performance of a linear model (multivariable logistic regression, GLM) with a non-linear model (random forest, RF) in a large national cohort. Methods: A retrospective analysis was performed on a multicenter database including 8902 critically ill patients with influenza A (H1N1)pdm09 or COVID-19 admitted to 184 Spanish ICUs. Demographic, clinical, laboratory, and microbiological data from the first 24 h were used. Prediction models were built using GLM and RF. The performance of the GLM was evaluated by area under the ROC curve (AUC), precision, sensitivity, and specificity, while the RF by out-of-bag (OOB) error and accuracy. In addition, in the RF, the im-portance of the variables in terms of accuracy reduction (AR) and Gini index reduction (GI) was determined. Results: Overall mortality in the ICU was 25.8%. Model performance was similar, with AUC = 76% for GLM, and AUC = 75.6% for RF. GLM identified 17 independent risk factors, while RF identified 19 for AR and 23 for GI. Thirteen variables were found to be important in both models. Laboratory variables such as procalcitonin, white blood cells, lactate, or D-dimer levels were not significant in GLM but were significant in RF. On the contrary, acute kidney injury and the presence of Acinetobacter spp. were important variables in the GLM but not in the RF. Conclusions: Although the performance of linear and non-linear models was similar, different risk factors were determined, depending on the model used. This alerts clinicians to the limitations and usefulness of studies limited to a single type of model.

Keywords: ICU mortality; generalized linear model; mortality risk factors; pandemic viruses; random forest.

Figure 1

Odds ratio (OR) plot of variables associated with ICU crude mortality in linear multivariate analysis (GLM). Abbreviations: cut: cut-off; APACHE II: Acute Physiology and Chronic Health Evaluation; SOFA: sequential organ failure assessment; AB: antibiotics; CPK: creatine phosphokinase; DD: D-dimer; MR_SA: methicillin-resistant S. aureus; MV: invasive mechanical ventilation; WBC: white blood cells; COPD: chronic obstructive pulmonary disease; dis: disfunction; Chr_Card_dis; chronic cardiac disease; HIV: human immunodeficiency virus; AKI: acute kidney injury; CRP: C-reactive protein; GAP_ICU_cut: time elapsed between diagnosing pandemic viral infection and admission to ICU; Chr_renal_dis: chronic renal disease; ID: immunosuppression; Rx-cutoff: > 2 fields with infiltrations in chest X-ray; PCT: procalcitonin; MS_SA: methicillin-sensitive S. aureus; GAP_diagnsosis_cut: time from symptoms onset to diagnosis; hematol_dis: hematologic disease; LDH: lactate dehydrogenase.

Figure 2

Contribution of each confounding variable according to the random forest (RF) model for variables associated with all-cause ICU mortality. Abbreviations: cut: cut-off; APACHE II: Acute Physiology and Chronic Health Evaluation; SOFA: sequential organ failure assessment; AB: antibiotics; CPK: creatine phosphokinase; DD: D-dimer; MR_SA: methicillin-resistant S. aureus; MV: invasive mechanical ventilation; WBC: white blood cells; COPD: chronic obstructive pulmonary disease; dis: disfunction; Chr_Card_dis; chronic cardiac disease; HIV: human immunodeficiency virus; AKI: acute kidney injury; CRP:C-reactive protein; GAP_ICU_cut: time elapsed between diagnosing pandemic viral infection and admission to ICU; Chr_renal_dis: chronic renal disease; ID: immunosuppression; Rx-cutoff: > 2 fields with infiltrations in chest X-ray; PCT: procalcitonin; MS_SA: methicillin-sensitive S. aureus; GAP_diagnsosis_cut: time from symptoms onset to diagnosis; hematol_dis: hematologic disease; LDH: lactate dehydrogenase).

Figure 3

Classification of patients according to the linear (generalized linear model—GLM) and non-linear (random forest—RF) models.

Figure 4

Distribution of the probability generated by each model (Class) with respect to the observed results (Real); (0 = survivors; 1 = non-survivors).