Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing

Abstract

The risk stratification of patients in the emergency department begins at triage. It is vital to stratify patients early based on their severity, since undertriage can lead to increased morbidity, mortality and costs. Our aim was to present a new approach to assist healthcare professionals at triage in the stratification of patients and in identifying those with higher risk of ICU admission. Adult patients assigned Manchester Triage System (MTS) or Emergency Severity Index (ESI) 1 to 3 from a Portuguese and a United States Emergency Departments were analyzed. Variables routinely collected at triage were used and natural language processing was applied to the patient chief complaint. Stratified random sampling was applied to split the data in train (70%) and test (30%) sets and 10-fold cross validation was performed for model training. Logistic regression, random forests, and a random undersampling boosting algorithm were used. We compared the performance obtained with the reference model-using only triage priorities-with the models using additional variables. For both hospitals, a logistic regression model achieved higher overall performance, yielding areas under the receiver operating characteristic and precision-recall curves of 0.91 (95% CI 0.90-0.92) and 0.30 (95% CI 0.27-0.33) for the United States hospital and of 0.85 (95% CI 0.83-0.86) and 0.06 (95% CI 0.05-0.07) for the Portuguese hospital. Heart rate, pulse oximetry, respiratory rate and systolic blood pressure were the most important predictors of ICU admission. Compared to the reference models, the models using clinical variables and the chief complaint presented higher recall for patients assigned MTS/ESI 3 and can identify patients assigned MTS/ESI 3 who are at risk for ICU admission.

Fig 1. Inclusion and exclusion criteria for Beth Israel Deaconess Medical Center dataset.

“n” corresponds to the number of triages.

Fig 2. Inclusion and exclusion criteria for Hospital Beatriz Ângelo dataset.

“n” corresponds to the number of triages.

Fig 3. Performance of regularized logistic regression in test using the different subsets of predictors for Beth Israel Deaconess Medical Center dataset.

ESI—Emergency Severity Index, AUROC—area under the ROC curve.

Fig 4. Relative importance of predictors of Intensive Care Unit admission for Beth Israel Deaconess Medical Center dataset obtained with regularized logistic regression using all available variables except triage priority.

Fig 5. Performance of regularized logistic regression in test using the different subsets of predictors for Hospital Beatriz Ângelo dataset.

MTS—Manchester Triage System, AUROC—area under the ROC curve.

Fig 6. Relative importance of predictors of Intensive Care Unit and Intermediate Care Unit admission for Hospital Beatriz Ângelo dataset, obtained with regularized logistic regression using all available variables except triage priority.

Exams are prescribed at the time of triage.

Fig 7. Performance of the model with isotonic calibration for Beth Israel Deaconess Medical Center (on the left) and of the multi-model with no calibration for Hospital Beatriz Ângelo (on the right) using in both logistic regression with all predictors except priority.

AUROC—Area under the ROC curve. AUPRC—Area under the precision recall curve.