Machine learning score to predict in-hospital outcomes in patients hospitalized in cardiac intensive care unit

Abstract

Aims: Although some scores based on traditional statistical methods are available for risk stratification in patients hospitalized in cardiac intensive care units (CICUs), the interest of machine learning (ML) methods for risk stratification in this field is not well established. We aimed to build an ML model to predict in-hospital major adverse events (MAE) in patients hospitalized in CICU.

Methods and results: In April 2021, a French national prospective multicentre study involving 39 centres included all consecutive patients admitted to CICU. The primary outcome was in-hospital MAE, including death, resuscitated cardiac arrest, or cardiogenic shock. Using 31 randomly assigned centres as an index cohort (divided into training and testing sets), several ML models were evaluated to predict in-hospital MAE. The eight remaining centres were used as an external validation cohort. Among 1499 consecutive patients included (aged 64 ± 15 years, 70% male), 67 had in-hospital MAE (4.3%). Out of 28 clinical, biological, ECG, and echocardiographic variables, seven were selected to predict MAE in the training set (n = 844). Boosted cost-sensitive C5.0 technique showed the best performance compared with other ML methods [receiver operating characteristic area under the curve (AUROC) = 0.90, precision-recall AUC = 0.57, F1 score = 0.5]. Our ML score showed a better performance than existing scores (AUROC: ML score = 0.90 vs. Thrombolysis In Myocardial Infarction (TIMI) score: 0.56, Global Registry of Acute Coronary Events (GRACE) score: 0.52, Acute Heart Failure (ACUTE-HF) score: 0.65; all P < 0.05). Machine learning score also showed excellent performance in the external cohort (AUROC = 0.88).

Conclusion: This new ML score is the first to demonstrate improved performance in predicting in-hospital outcomes over existing scores in patients admitted to the intensive care unit based on seven simple and rapid clinical and echocardiographic variables.

Trial registration: ClinicalTrials.gov Identifier: NCT05063097.

Keywords: Death; Echocardiography; Intensive cardiac care unit; Machine learning; Outcomes.

Graphical Abstract

Figure 1

Flow chart of the study population.

Figure 2

Feature selection using Boruta technique. The Boruta variable selection algorithm was applied to obtain the variables that contribute the most to the prediction of major adverse events. Considering a confidence level with a P-value threshold of 0.05, seven variables were considered important: tricuspid annular plane systolic excursion, carbon monoxide level, mean arterial pressure, left ventricular ejection fraction, Killip class, and ratio E/e′.

Figure 3

Decision tree illustrating the machine learning model. Boosted C5.0 model is based on the idea of adaptive boosting. The main objective is to combine several weak classifiers into a stronger one, while keeping the robustness to overfitting of the weak classifiers. The classification tree with the lowest error (3.4%) is shown below.

Figure 4

Compared performances of our machine learning model and existing risk scores. (A) shows the receiver operating curves and (B) shows the precision–recall curves comparing the performances of our machine learning model, logistic regression (including the same variables as the machine learning model), a traditional model (clinical a priori regression model), and other existing scores in the internal validation data set of the index cohort. The machine learning model had a significantly higher area under the curve for receiver operating curves and precision–recall curves in predicting in-hospital major adverse events than all other risk models (P < 0.001). The results are given with their 95% confidence interval.