Early Prediction of Hemodynamic Shock in Pediatric Intensive Care Units With Deep Learning on Thermal Videos

Abstract

Shock is one of the major killers in intensive care units, and early interventions can potentially reverse it. In this study, we advance a noncontact thermal imaging modality for continuous monitoring of hemodynamic shock working on 1,03,936 frames from 406 videos recorded longitudinally upon 22 pediatric patients. Deep learning was used to preprocess and extract the Center-to-Peripheral Difference (CPD) in temperature values from the videos. This time-series data along with the heart rate was finally analyzed using Long-Short Term Memory models to predict the shock status up to the next 6 h. Our models achieved the best area under the receiver operating characteristic curve of 0.81 ± 0.06 and area under the precision-recall curve of 0.78 ± 0.05 at 5 h, providing sufficient time to stabilize the patient. Our approach, thus, provides a reliable shock prediction using an automated decision pipeline that can provide better care and save lives.

Keywords: ICU—intensive care unit; artificial intelligence; computer vision; deep learning; hemodynamic shock; thermal imaging.

FIGURE 1

Shock prediction steps. The summary of the shock pipeline shows the steps from video frame extraction to shock prediction. Step 1 comprises sampling videos to extract frames. Step 2 classifies frames into covered or uncovered, while also finding the presence of multiple people in the frame and mask them, to avoid confusion. The masked images are then input to the ResUNet based segmentation model and CPD is hence extracted. The series of CPDs are then passed through a time-series sequence classifier, and finally, the predictions are made for shock for the next 6 h.

FIGURE 2

Shock prediction pipeline. Illustration of the pipeline followed for the detection and prediction of shock and no-shock. Each frame of the video was examined for uncovered/covered. The uncovered frames were then filtered off the presence of people other than the patient present in the frame. The frame was finally passed from the segmentation and CPD extraction model, further collecting the sequences used for LSTM time-series classification. The time sequences had CPD and heart rate as features and an appropriate window length of 256 s was chosen. Since the data were highly imbalanced, the SMOTE upsampling method was used in training the LSTM model. The detection and prediction of shock were carried out at 0 h and for the next 6 h, respectively.

FIGURE 3

Quality assessment of models. The F1 score comparison for the three models (i.e., Linear Mixed-Effects model, Random Forest, and LSTM) tested on (A) CPD and heart rate and (B) CPD parameter only. It can be observed that the LSTM model outperforms the other two models in both cases, making it the primary choice of this research. The rest of the comparison plots are depicted in Supplementary Figure S1.

FIGURE 4

Quality evaluation on the LSTM model. (A) Quality evaluation of the LSTM time-series classification models at lead points of up to 6 h. The best performance of (B) AUPRC and (C) AUROC was obtained at a 5-h prediction. The rest of the performance metric results are demonstrated in Supplementary Figure S2. The results of all are shown in Table 2. The standard error (SE) for each is calculated from cross-validation, by taking k = 10.