Model for Predicting In-Hospital Mortality of Physical Trauma Patients Using Artificial Intelligence Techniques: Nationwide Population-Based Study in Korea

Abstract

Background: Physical trauma-related mortality places a heavy burden on society. Estimating the mortality risk in physical trauma patients is crucial to enhance treatment efficiency and reduce this burden. The most popular and accurate model is the Injury Severity Score (ISS), which is based on the Abbreviated Injury Scale (AIS), an anatomical injury severity scoring system. However, the AIS requires specialists to code the injury scale by reviewing a patient's medical record; therefore, applying the model to every hospital is impossible.

Objective: We aimed to develop an artificial intelligence (AI) model to predict in-hospital mortality in physical trauma patients using the International Classification of Disease 10th Revision (ICD-10), triage scale, procedure codes, and other clinical features.

Methods: We used the Korean National Emergency Department Information System (NEDIS) data set (N=778,111) compiled from over 400 hospitals between 2016 and 2019. To predict in-hospital mortality, we used the following as input features: ICD-10, patient age, gender, intentionality, injury mechanism, and emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and procedure codes. We proposed the ensemble of deep neural networks (EDNN) via 5-fold cross-validation and compared them with other state-of-the-art machine learning models, including traditional prediction models. We further investigated the effect of the features.

Results: Our proposed EDNN with all features provided the highest area under the receiver operating characteristic (AUROC) curve of 0.9507, outperforming other state-of-the-art models, including the following traditional prediction models: Adaptive Boosting (AdaBoost; AUROC of 0.9433), Extreme Gradient Boosting (XGBoost; AUROC of 0.9331), ICD-based ISS (AUROC of 0.8699 for an inclusive model and AUROC of 0.8224 for an exclusive model), and KTAS (AUROC of 0.1841). In addition, using all features yielded a higher AUROC than any other partial features, namely, EDNN with the features of ICD-10 only (AUROC of 0.8964) and EDNN with the features excluding ICD-10 (AUROC of 0.9383).

Conclusions: Our proposed EDNN with all features outperforms other state-of-the-art models, including the traditional diagnostic code-based prediction model and triage scale.

Keywords: Information system; artificial intelligence; deep neural network; emergency department; injury; international classification of disease; mortality prediction; prediction model; severity score; trauma.

Figure 1

Flowchart of the patient selection process.

Figure 2

Process flow of our artificial intelligence (AI) model development: data, deep neural network (DNN) architecture, ensemble DNN model, and performance comparison. AdaBoost: Adaptive Boosting; EMR-ISS: mortality ratio-adjusted Injury Severity Score; ICD: International Classification of Diseases; KTAS: Korean Triage and Acuity Scale; LightGBM: light gradient boosting machine; SRR: survival risk ratio; XGBoost: Extreme Gradient Boosting.

Figure 3

Results of the ranked normalized feature importance from the Adaptive Boosting (AdaBoost) model. KTAS: Korean Triage and Acuity Scale.

Figure 4

Area under the receiver operating characteristic (AUROC) curves for our model, Adaptive Boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), and light gradient boosting machine (LightGBM): (left) using all 914 features including International Classification of Diseases 10th Revision (ICD-10), (middle) using 48 features excluding ICD-10, and (right) using 866 features with ICD-10 only. DNN: deep neural network.

Figure 5

Area under the receiver operating characteristic (AUROC) curves of our model and 4 traditional models. AUC: area under the curve; DNN: deep neural network; EMR-ISS: mortality ratio-adjusted Injury Severity Score; KTAS: Korean Triage and Acuity Scale; SRR: survival risk ratio.