Continuous Prediction of Mortality in the PICU: A Recurrent Neural Network Model in a Single-Center Dataset

Abstract

Objectives: Develop, as a proof of concept, a recurrent neural network model using electronic medical records data capable of continuously assessing an individual child's risk of mortality throughout their ICU stay as a proxy measure of severity of illness.

Design: Retrospective cohort study.

Setting: PICU in a tertiary care academic children's hospital.

Patients/subjects: Twelve thousand five hundred sixteen episodes (9,070 children) admitted to the PICU between January 2010 and February 2019, partitioned into training (50%), validation (25%), and test (25%) sets.

Interventions: None.

Measurements and main results: On 2,475 test set episodes lasting greater than or equal to 24 hours in the PICU, the area under the receiver operating characteristic curve of the recurrent neural network's 12th hour predictions was 0.94 (CI, 0.93-0.95), higher than those of Pediatric Index of Mortality 2 (0.88; CI, [0.85-0.91]; p < 0.02), Pediatric Risk of Mortality III (12th hr) (0.89; CI, [0.86-0.92]; p < 0.05), and Pediatric Logistic Organ Dysfunction day 1 (0.85; [0.81-0.89]; p < 0.002). The recurrent neural network's discrimination increased with more acquired data and smaller lead time, achieving a 0.99 area under the receiver operating characteristic curve 24 hours prior to discharge. Despite not having diagnostic information, the recurrent neural network performed well across different primary diagnostic categories, generally achieving higher area under the receiver operating characteristic curve for these groups than the other three scores. On 692 test set episodes lasting greater than or equal to 5 days in the PICU, the recurrent neural network area under the receiver operating characteristic curves significantly outperformed their daily Pediatric Logistic Organ Dysfunction counterparts (p < 0.005).

Conclusions: The recurrent neural network model can process hundreds of input variables contained in a patient's electronic medical record and integrate them dynamically as measurements become available. Its high discrimination suggests the recurrent neural network's potential to provide an accurate, continuous, and real-time assessment of a child in the ICU.

Figure 1.

Overview of the recurrent neural network (RNN) model with its inputs (denoted by x) and outputs (denoted by y). Note that the RNN model is a many-to-many model that generates an output at every timestep where there is an input. t = time.

Figure 2.

Recurrent neural network (RNN) area under the receiver operating characteristic curve (AUROC), as a function of time relative to ICU admission (A) or ICU discharge (B), on test set episodes whose length of stay (LOS) was at least 24 hr and had Pediatric Logistic Organ Dysfunction (PELOD) day 1, Pediatric Index of Mortality (PIM) 2, and Pediatric Risk of Mortality (PRISM) III scores.

Figure 3.

Area under the receiver operating characteristic curve (AUROC), as a function of days since ICU admission, of recurrent neural network (RNN) predictions and Pediatric Logistic Organ Dysfunction (PELOD) daily scores on test set episodes whose length of stay (LOS) was at least 5 d and had Pediatric Index of Mortality (PIM) 2 and Pediatric Risk of Mortality (PRISM) III scores.

Figure 4.

Calibration of recurrent neural network predictions at all 721,024 time points of all test set episodes. Each of the 50 quantiles contains either 13,865 or 13,866 predictions.

Figure 5.

Recurrent neural network–generated mortality risks, as functions of time, for two surviving episodes (cyan and green) and two nonsurviving ones (purple and orange).