Predicting Prolonged Apnea During Nurse-Administered Procedural Sedation: Machine Learning Study

Abstract

Background: Capnography is commonly used for nurse-administered procedural sedation. Distinguishing between capnography waveform abnormalities that signal the need for clinical intervention for an event and those that do not indicate the need for intervention is essential for the successful implementation of this technology into practice. It is possible that capnography alarm management may be improved by using machine learning to create a "smart alarm" that can alert clinicians to apneic events that are predicted to be prolonged.

Objective: To determine the accuracy of machine learning models for predicting at the 15-second time point if apnea will be prolonged (ie, apnea that persists for >30 seconds).

Methods: A secondary analysis of an observational study was conducted. We selected several candidate models to evaluate, including a random forest model, generalized linear model (logistic regression), least absolute shrinkage and selection operator regression, ridge regression, and the XGBoost model. Out-of-sample accuracy of the models was calculated using 10-fold cross-validation. The net benefit decision analytic measure was used to assist with deciding whether using the models in practice would lead to better outcomes on average than using the current default capnography alarm management strategies. The default strategies are the aggressive approach, in which an alarm is triggered after brief periods of apnea (typically 15 seconds) and the conservative approach, in which an alarm is triggered for only prolonged periods of apnea (typically >30 seconds).

Results: A total of 384 apneic events longer than 15 seconds were observed in 61 of the 102 patients (59.8%) who participated in the observational study. Nearly half of the apneic events (180/384, 46.9%) were prolonged. The random forest model performed the best in terms of discrimination (area under the receiver operating characteristic curve 0.66) and calibration. The net benefit associated with the random forest model exceeded that associated with the aggressive strategy but was lower than that associated with the conservative strategy.

Conclusions: Decision curve analysis indicated that using a random forest model would lead to a better outcome for capnography alarm management than using an aggressive strategy in which alarms are triggered after 15 seconds of apnea. The model would not be superior to the conservative strategy in which alarms are only triggered after 30 seconds.

Keywords: anaesthesia; anesthesia; apnea; apnoea; capnography; conscious sedation; informatics; machine learning; medical informatics; nursing; patient safety; procedural sedation and analgesia; sedation; sleep apnea.

Figure 1

Area under the receiver operating characteristics curve. LASSO, least absolute shrinkage and selection operator.

Figure 2

Threshold performance plot for all models evaluated. LASSO, least absolute shrinkage and selection operator; PPV, positive predictive value; NPV, negative predictive value.

Figure 3

Calibration plot for all models evaluated. LASSO, least absolute shrinkage and selection operator.

Figure 4

Decision curve analysis plots.