Supervised machine learning model to predict mortality in patients undergoing venovenous extracorporeal membrane oxygenation from a nationwide multicentre registry

Abstract

Background: Existing models have performed poorly when predicting mortality for patients undergoing venovenous extracorporeal membrane oxygenation (VV-ECMO). This study aimed to develop and validate a machine learning (ML)-based prediction model to predict 90-day mortality in patients undergoing VV-ECMO.

Methods: This study included 368 patients with acute respiratory failure undergoing VV-ECMO from 16 tertiary hospitals across South Korea between 2012 and 2015. The primary outcome was the 90-day mortality after ECMO initiation. The inputs included all available features (n=51) and those from the electronic health record (EHR) systems without preprocessing (n=40). The discriminatory strengths of ML models were evaluated in both internal and external validation sets. The models were compared with conventional models, such as respiratory ECMO survival prediction (RESP) and predicting death for severe acute respiratory distress syndrome on VV-ECMO (PRESERVE).

Results: Extreme gradient boosting (XGB) (areas under the receiver operating characteristic curve, AUROC 0.82, 95% CI (0.73 to 0.89)) and light gradient boosting (AUROC 0.81 (95% CI 0.71 to 0.88)) models achieved the highest performance using EHR's and all other available features. The developed models had higher AUROCs (95% CI 0.76 to 0.82) than those of RESP (AUROC 0.66 (95% CI 0.56 to 0.76)) and PRESERVE (AUROC 0.71 (95% CI 0.61 to 0.81)). Additionally, we achieved an AUROC (0.75) for 90-day mortality in external validation in the case of the XGB model, which was higher than that of RESP (0.70) and PRESERVE (0.67) in the same validation dataset.

Conclusions: ML prediction models outperformed previous mortality risk models. This model may be used to identify patients who are unlikely to benefit from VV-ECMO therapy during patient selection.

Keywords: ARDS; critical care.

Figure 1

Discrimination performance of prediction models with EHR features for 90-day mortality in the interval validation set. AUROC, area under receiver operating characteristics; EHR, electronic health record; LGB, light gradient boosting; LR, logistic regression; MLP, multilayer perceptron; RF, random forest; SVM, support vector machine; XGB, extreme gradient boosting.

Figure 2

ROC comparing 90-day mortality prediction models using EHR features with the RESP and PRESERVE scores in the internal validation set. ECMO, extracorporeal membrane oxygenation; PRESERVE, predicting death for severe acute respiratory distress syndrome on VV-ECMO; RESP, respiratory ECMO survival prediction; ROC, receiver operating characteristics; XGB, extreme gradient boosting.

Figure 3

SHAP analysis of 90-day mortality prediction with EHR features in the internal validation set. The colour scheme in the plot uses red to represent higher features values and blue to represent lower feature values. On the x-axis, positive values indicate an increased risk of mortality, while negative values represent a decreased risk of mortality. EHR, electronic health record; SHAP, shapley additive explanations.

Figure 4

Calibration performance of 90-day mortality prediction models with EHR features in the internal validation set. BSL, Brier Score Loss; EHR, electronic health record; LGB, light gradient boosting; LR, logistic regression; MLP, multilayer perceptron; RF, random forest; SVM, support vector machine.