Machine learning-based prediction of critical illness in children visiting the emergency department

Abstract

Objectives: Triage is an essential emergency department (ED) process designed to provide timely management depending on acuity and severity; however, the process may be inconsistent with clinical and hospitalization outcomes. Therefore, studies have attempted to augment this process with machine learning models, showing advantages in predicting critical conditions and hospitalization outcomes. The aim of this study was to utilize nationwide registry data to develop a machine learning-based classification model to predict the clinical course of pediatric ED visits.

Methods: This cross-sectional observational study used data from the National Emergency Department Information System on emergency visits of children under 15 years of age from January 1, 2016, to December 31, 2017. The primary and secondary outcomes were to identify critically ill children and predict hospitalization from triage data, respectively. We developed and tested a random forest model with the under sampled dataset and validated the model using the entire dataset. We compared the model's performance with that of the conventional triage system.

Results: A total of 2,621,710 children were eligible for the analysis and included 12,951 (0.5%) critical outcomes and 303,808 (11.6%) hospitalizations. After validation, the area under the receiver operating characteristic curve was 0.991 (95% confidence interval [CI] 0.991-0.992) for critical outcomes and 0.943 (95% CI 0.943-0.944) for hospitalization, which were higher than those of the conventional triage system.

Conclusions: The machine learning-based model using structured triage data from a nationwide database can effectively predict critical illness and hospitalizations among children visiting the ED.

Fig 1. Receiver operating characteristic curves comparing the performance of the random forest model with that of the conventional triage system (pedKTAS).

A. Under sampled dataset—critical cases, B. Validation with the entire dataset compared with pedKTAS–critical cases, C. Under sampled dataset–hospitalization, D. Validation with the entire dataset compared with pedKTAS–hospitalization. AUC = Area under the curve, CI = Confidence interval.

Fig 2. Precision-recall curves comparing the performance of the random forest model with that of the conventional triage system (pedKTAS).

A. Under sampled dataset—critical cases, B. Validation with the entire dataset compared with pedKTAS–critical cases, C. Under sampled dataset–hospitalization, D. Validation with the entire dataset compared with pedKTAS–hospitalization. AUC = Area under the curve, CI = Confidence interval.

Fig 3. Top thirty predictors with the highest importance for each outcome.

The importance of each feature was calculated through information gain using the difference in Gini impurity reduction. The “feature importance” function of Python’s scikit-learn library was used [34]. A. Critical cases, and B. hospitalization.