An Artificial Intelligence Model for Predicting Trauma Mortality Among Emergency Department Patients in South Korea: Retrospective Cohort Study

Abstract

Background: Within the trauma system, the emergency department (ED) is the hospital's first contact and is vital for allocating medical resources. However, there is generally limited information about patients that die in the ED.

Objective: The aim of this study was to develop an artificial intelligence (AI) model to predict trauma mortality and analyze pertinent mortality factors for all patients visiting the ED.

Methods: We used the Korean National Emergency Department Information System (NEDIS) data set (N=6,536,306), incorporating over 400 hospitals between 2016 and 2019. We included the International Classification of Disease 10th Revision (ICD-10) codes and chose the following input features to predict ED patient mortality: age, sex, intentionality, injury, emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and vital signs. We compared three different feature set performances for AI input: all features (n=921), ICD-10 features (n=878), and features excluding ICD-10 codes (n=43). We devised various machine learning models with an ensemble approach via 5-fold cross-validation and compared the performance of each model with that of traditional prediction models. Lastly, we investigated explainable AI feature effects and deployed our final AI model on a public website, providing access to our mortality prediction results among patients visiting the ED.

Results: Our proposed AI model with the all-feature set achieved the highest area under the receiver operating characteristic curve (AUROC) of 0.9974 (adaptive boosting [AdaBoost], AdaBoost + light gradient boosting machine [LightGBM]: Ensemble), outperforming other state-of-the-art machine learning and traditional prediction models, including extreme gradient boosting (AUROC=0.9972), LightGBM (AUROC=0.9973), ICD-based injury severity scores (AUC=0.9328 for the inclusive model and AUROC=0.9567 for the exclusive model), and KTAS (AUROC=0.9405). In addition, our proposed AI model outperformed a cutting-edge AI model designed for in-hospital mortality prediction (AUROC=0.7675) for all ED visitors. From the AI model, we also discovered that age and unresponsiveness (coma) were the top two mortality predictors among patients visiting the ED, followed by oxygen saturation, multiple rib fractures (ICD-10 code S224), painful response (stupor, semicoma), and lumbar vertebra fracture (ICD-10 code S320).

Conclusions: Our proposed AI model exhibits remarkable accuracy in predicting ED mortality. Including the necessity for external validation, a large nationwide data set would provide a more accurate model and minimize overfitting. We anticipate that our AI-based risk calculator tool will substantially aid health care providers, particularly regarding triage and early diagnosis for trauma patients.

Keywords: ICD; artificial intelligence; cohort; death; emergency; emergency department; international classification of disease; model; models; mortality; mortality prediction; national; nationwide; predict; prediction; predictive; retrospective; trauma; traumatic.

Figure 1

Patient selection process. ED: emergency department; S: International Classification of Diseases (ICD) code to signify trauma in a single body region; T: ICD code to signify trauma in multiple or unspecified regions.

Figure 2

Ranked normalized feature importance from the selected AdaBoost model. KTAS: Korean Triage and Acuity Scale. See Table S5 in Multimedia Appendix 1 for the definition of "S" and "T" International Classification of Diseases 10th revision codes.

Figure 3

Receiver operating characteristic curves for the (left) selected adaptive boosting (AdaBoost) and three traditional models and (right) relative AdaBoost features. AUROC: area under the receiver operating characteristic curve; ICD-10: International Classification of Diseases 10th revision; KTAS: Korean Triage and Acuity Scale; SRR: survival risk ratio.

Figure 4

A new pipeline for predicting emergency department (ED) and in-hospital mortality [4,5]. AI: artificial intelligence; AIS: Abbreviated Injury Scale; ICD-10: International Classification for Diseases 10th revision; ICU: intensive care unit; OR: operating room.