Predicting 30-day mortality of patients with pneumonia in an emergency department setting using machine-learning models

Abstract

Objective: This study aimed to confirm the accuracy of a machine-learning-based model in predicting the 30-day mortality of patients with pneumonia and evaluating whether they were required to be admitted to the intensive care unit (ICU).

Methods: The study conducted a retrospective analysis of pneumonia patients at an emergency department (ED) in Seoul, Korea, from January 1, 2016 to December 31, 2017. Patients aged 18 years or older with a pneumonia registry designation on their electronic medical record were enrolled. We collected their demographic information, mental status, and laboratory findings. Three models were used: the pre-existing CURB-65 model, and the CURB-RF and Extensive CURB-RF models, which were machine-learning models that used a random forest algorithm. The primary outcomes were ICU admission from the ED or 30-day mortality. Receiver operating characteristic curves were constructed for the models, and the areas under these curves were compared.

Results: Out of the 1,974 pneumonia patients, 1,732 patients were eligible to be included in the study; from these, 473 patients died within 30 days or were initially admitted to the ICU from the ED. The area under receiver operating characteristic curves of CURB-65, CURB-RF, and extensive-CURB-RF were 0.615 (0.614-0.616), 0.701 (0.700-0.702), and 0.844 (0.843-0.845), respectively.

Conclusion: The proposed machine-learning models could predict the mortality of patients with pneumonia more accurately than the pre-existing CURB-65 model and can help decide whether the patient should be admitted to the ICU.

Keywords: Emergency service, hospital; Machine-learning; Mortality; Pneumonia.

Fig. 1.

Schematic showing how the data were processed in this study. By choosing a random under-sampled selection of the control group at a 1:2 ratio, the presence of an imbalanced outcome variable was solved. The dataset was divided into two sets—0.7 for the training set and 0.3 for the test set. The training set went through ten-fold cross-validation. After learning the data of the training set using a random forest algorithm, the best model derived was evaluated in the test set. This process was repeated 1,000 times.

Fig. 2.

Patient selection process and the number of patients distributed in each group. The case group comprising death within 30 days or intensive care unit admission from the emergency department (ED) involved 473 patients. Through under-sampled selection, three times as many patients were selected. EMR, electronic medical record.

Fig. 3.

Comparison of receiver operating characteristics curves among the three models.