Machine learning-based mortality prediction model for heat-related illness

Abstract

In this study, we aimed to develop and validate a machine learning-based mortality prediction model for hospitalized heat-related illness patients. After 2393 hospitalized patients were extracted from a multicentered heat-related illness registry in Japan, subjects were divided into the training set for development (n = 1516, data from 2014, 2017-2019) and the test set (n = 877, data from 2020) for validation. Twenty-four variables including characteristics of patients, vital signs, and laboratory test data at hospital arrival were trained as predictor features for machine learning. The outcome was death during hospital stay. In validation, the developed machine learning models (logistic regression, support vector machine, random forest, XGBoost) demonstrated favorable performance for outcome prediction with significantly increased values of the area under the precision-recall curve (AUPR) of 0.415 [95% confidence interval (CI) 0.336-0.494], 0.395 [CI 0.318-0.472], 0.426 [CI 0.346-0.506], and 0.528 [CI 0.442-0.614], respectively, compared to that of the conventional acute physiology and chronic health evaluation (APACHE)-II score of 0.287 [CI 0.222-0.351] as a reference standard. The area under the receiver operating characteristic curve (AUROC) values were also high over 0.92 in all models, although there were no statistical differences compared to APACHE-II. This is the first demonstration of the potential of machine learning-based mortality prediction models for heat-related illnesses.

Figure 1

Flow diagram of patient inclusion procedure.

Figure 2

(A) Absolute values of standardized beta coefficients for the logistic regression model. (B) Feature importances of variables for the random forest model. (C) Feature importances of variables for the XGBoost model. Asterisk shows the feature in a positive correlation to the survival outcome. Location (outdoor/indoor)* and gender* refer to outdoor location and male are positive correlation to the survival outcome, respectively. GCS Glasgow coma scale, AST aspartate aminotransferase, ALT alanine aminotransferase, SpO₂ oxygen saturation, BUN blood urea nitrogen, PT-INR prothrombin time/international normalized ratio.

Figure 3

Comparison of ROC curves, PR curves, AUROC, and AUPR among the developed machine-learning models and APACHE-II score for mortality prediction. ROC Receiver operating characteristic, PR precision-recall, AUROC area under the receiver operating characteristic curve, AUPR area under the precision-recall curve, APACHE acute physiology and chronic health evaluation, CI confidence interval.