Cardiorespiratory signature of neonatal sepsis: development and validation of prediction models in 3 NICUs

Abstract

Background: Heart rate characteristics aid early detection of late-onset sepsis (LOS), but respiratory data contain additional signatures of illness due to infection. Predictive models using cardiorespiratory data may improve early sepsis detection. We hypothesized that heart rate (HR) and oxygenation (SpO₂) data contain signatures that improve sepsis risk prediction over HR or demographics alone.

Methods: We analyzed cardiorespiratory data from very low birth weight (VLBW, <1500 g) infants admitted to three NICUs. We developed and externally validated four machine learning models to predict LOS using features calculated every 10 m: mean, standard deviation, skewness, kurtosis of HR and SpO₂, and cross-correlation. We compared feature importance, discrimination, calibration, and dynamic prediction across models and cohorts. We built models of demographics and HR or SpO₂ features alone for comparison with HR-SpO₂ models.

Results: Performance, feature importance, and calibration were similar among modeling methods. All models had favorable external validation performance. The HR-SpO₂ model performed better than models using either HR or SpO₂ alone. Demographics improved the discrimination of all physiologic data models but dampened dynamic performance.

Conclusions: Cardiorespiratory signatures detect LOS in VLBW infants at 3 NICUs. Demographics risk-stratify, but predictive modeling with both HR and SpO₂ features provides the best dynamic risk prediction.

Impact: Heart rate characteristics aid early detection of late-onset sepsis, but respiratory data contain signatures of illness due to infection. Predictive models using both heart rate and respiratory data may improve early sepsis detection. A cardiorespiratory early warning score, analyzing heart rate from electrocardiogram or pulse oximetry with SpO₂, predicts late-onset sepsis within 24 h across multiple NICUs and detects sepsis better than heart rate characteristics or demographics alone. Demographics risk-stratify, but predictive modeling with both HR and SpO₂ features provides the best dynamic risk prediction. The results increase understanding of physiologic signatures of neonatal sepsis.

Fig 1.. Schematic Overview of Methods.

From top to bottom, we processed the raw signals, sampled at 0.5 Hz, by calculating HR and SpO₂ features every ten minutes. Each 10-minute window, from 72 hours after birth until NICU discharge or death, was labeled as late-onset sepsis (LOS), control, or removed as a blackout period window. Data from NICU 1 were used to train four machine learning models. Before external validation on data at NICU 2 & 3, post-processing steps included smoothing the 10-minute model outputs over 4 hours and recalibrating. Metrics used for external validation included discrimination by AUC, calibration, and plotting the average risk over the 48 hours preceding sepsis to look for a dynamic rise from baseline near the time of diagnosis by blood culture.

Fig 2.. Calibration plots.

Calibration of each POWS model for (A) NICU 1, (B) NICU 2, and (C) NICU 3. Predicted risk relative to average is on the abscissa and observed risk relative to average is on the ordinate. Each point represents one decile of predicted risk. The line of identity is shown as a dashed line. LR = logistic regression, NN = neural network, XG = XGBoost, RF = random forest

Fig 3.. The Average Risk of Sepsis

The average relative risk of sepsis as predicted by each model as a function of the time to event in hours. Panels show the results of each model at (A) NICU 1, (B) NICU 2, and (C) NICU 3. Results are shown for the four POWS models and the demographic-only model (in gray). Black crosses indicate times where the model outputs are significantly higher (p < 0.05) than outputs from the same patient 24 h prior. LR = logistic regression, NN = neural network, XG = XGBoost, RF = random forest

Fig 4.. Variable Importance Plots.

Variable importance plots for components of the logistic regression (LR), neural net (NN), XG Boost (XG), and random forest (RF) models. Features are ordered by decreasing AUC loss introduced by permuting the values of each feature.

Fig 5.. Evaluating the Model Sensitivity across a Range of Thresholds.

Using an alert strategy for physiologic-based models where an alarm is initially triggered based on a threshold crossing and the alarm remains on until 24 consecutive hours where there has been no threshold crossings, we selected a range of thresholds and then calculated the number of alerts per day and required an alert display in the 3 days preceding the clinical diagnosis of sepsis. We defined alerts as daily threshold crossings for the demographic model. The y-axis displays the percent of sepsis events detected when allowing for different numbers of alerts per day.