The Sleep Well Baby project: an automated real-time sleep-wake state prediction algorithm in preterm infants

Abstract

Study objectives: Sleep is an important driver of early brain development. However, sleep is often disturbed in preterm infants admitted to the neonatal intensive care unit (NICU). We aimed to develop an automated algorithm based on routinely measured vital parameters to classify sleep-wake states of preterm infants in real-time at the bedside.

Methods: In this study, sleep-wake state observations were obtained in 1-minute epochs using a behavioral scale developed in-house while vital signs were recorded simultaneously. Three types of vital parameter data, namely, heart rate, respiratory rate, and oxygen saturation, were collected at a low-frequency sampling rate of 0.4 Hz. A supervised machine learning workflow was used to train a classifier to predict sleep-wake states. Independent training (n = 37) and validation datasets were validation n = 9) datasets were used. Finally, a setup was designed for real-time implementation at the bedside.

Results: The macro-averaged area-under-the-receiver-operator-characteristic (AUROC) of the automated sleep staging algorithm ranged between 0.69 and 0.82 for the training data, and 0.61 and 0.78 for the validation data. The algorithm provided the most accurate prediction for wake states (AUROC = 0.80). These findings were well validated on an independent sample (AUROC = 0.77).

Conclusions: With this study, to the best of our knowledge, a reliable, nonobtrusive, and real-time sleep staging algorithm was developed for the first time for preterm infants. Deploying this algorithm in the NICU environment may assist and adapt bedside clinical work based on infants' sleep-wake states, potentially promoting the early brain development and well-being of preterm infants.

Keywords: automated sleep staging; machine learning; neonatal intensive care; preterm; sleep.

Figure 1.

Schematic representation of the machine-learning model development. The nested cross-validation procedure was applied to select the random-forest classifier over other models and identify its optimal hyperparameters. The regular cross-validation procedure was applied to approximate out-of-sample performance using the training dataset, which was later validated using the validation dataset.

Figure 2.

AUROCs calculated for the random forest classifier using (A) the regular cross-validation procedure on the training dataset and (B) the validation dataset. Performance mean and 95% Confidence Intervals (solid lines and filled areas, respectively) were calculated using 250-fold bootstrapping. Left, macroaveraged over all sleep–wake states; right, for the individual sleep–wake states.

Figure 3.

Top panels: model calibration per sleep–wake state (W = Wake, AS = Active Sleep, QS = Quiet Sleep), with data points split over 20 quantiles (bins) per sleep–wake state, for (A) training dataset and (B) validation dataset. The horizontal axis indicates the mean value of all predicted probability in each bin and the vertical axis indicates the proportion of positive class samples in each bin. Bottom panels: predicted time spent in a sleep–wake state (horizontal axis) versus the observed time in the sleep–wake state (vertical axis). Dots represent individual patients and are colored by sleep–wake state (Green/W = Wake, Orange/AS = Active Sleep, Red/QS = Quiet Sleep). Predicted time corresponds to summation of all model probability predictions. Panel (C) shows results for the training dataset. Panel (D) shows the results for the validation dataset.

Figure 4.

SHAP values of top-10 features of Active sleep (top left), Quiet sleep (top right) and Wake (bottom left). Naming convention is _____. Dots represent individual data points. Feature values are indicated on a red-blue scale, with red representing a high value for that specific feature. Given the feature and its value for a specific data point, the impact on the model output (SHAP-value) is depicted on the x-axis.

Figure 5.

Bedside implementation architecture. Every minute, the BedBase application extracts real-time patient information and sends a POST request to the SWB API. Other than age, no identifiable data leaves the NICU. Before each prediction the SWB application assures that the data is valid (e.g. no more than 50% missing values per parameter in each time window, range of allowed values per parameter, etc.) and checks that the patient age falls within the range the algorithm was designed for (28–34 weeks postmenstrual age). If all criteria are met a sleep-state prediction is made and the result returned. Aggregated performance metrics are logged, e.g. ratio of wake predictions per day, such that an ICT and data scientists can monitor performance without compromising patient privacy. Data and code version control is implemented to ensure the SWB API always uses the correct model version from the model repository.