Dynamic individual vital sign trajectory early warning score (DyniEWS) versus snapshot national early warning score (NEWS) for predicting postoperative deterioration

Abstract

Aims: International early warning scores (EWS) including the additive National Early Warning Score (NEWS) and logistic EWS currently utilise physiological snapshots to predict clinical deterioration. We hypothesised that a dynamic score including vital sign trajectory would improve discriminatory power.

Methods: Multicentre retrospective analysis of electronic health record data from postoperative patients admitted to cardiac surgical wards in four UK hospitals. Least absolute shrinkage and selection operator-type regression (LASSO) was used to develop a dynamic model (DyniEWS) to predict a composite adverse event of cardiac arrest, unplanned intensive care re-admission or in-hospital death within 24 h.

Results: A total of 13,319 postoperative adult cardiac patients contributed 442,461 observations of which 4234 (0.96%) adverse events in 24 h were recorded. The new dynamic model (AUC = 0.80 [95% CI 0.78-0.83], AUPRC = 0.12 [0.10-0.14]) outperforms both an updated snapshot logistic model (AUC = 0.76 [0.73-0.79], AUPRC = 0.08 [0.60-0.10]) and the additive National Early Warning Score (AUC = 0.73 [0.70-0.76], AUPRC = 0.05 [0.02-0.08]). Controlling for the false alarm rates to be at current levels using NEWS cut-offs of 5 and 7, DyniEWS delivers a 7% improvement in balanced accuracy and increased sensitivities from 41% to 54% at NEWS 5 and 18% to -30% at NEWS 7.

Conclusions: Using an advanced statistical approach, we created a model that can detect dynamic changes in risk of unplanned readmission to intensive care, cardiac arrest or in-hospital mortality and can be used in real time to risk-prioritise clinical workload.

Keywords: Cardiac surgery; Dynamic prediction; Early warning scores; National early warning score; Postoperative deterioration.

Fig. 1

Flow chart of data processing steps for vitalPAC™.

Fig. 2

Assessments of model performance on training and test data. (A) Calibration plot for internal temporal validation using training data (the number of observations = 405,692, the number of patients = 12,307). (B) (Left): Receiver-operating characteristic curves for fitting each method to the test data (the number of observations = 36,769, the number of patients = 1,150, the reference random-classification gives a 45-degree straight line with area under the curve at 50%). (B) (Right): Precision-recall curves for fitting each method to the test data (the reference is the horizontal line with precision equal to the prevalence of adverse events, 0.93%). P denotes the number of adverse events and N denotes the number of non-events.