A Trend-Based Early Warning Score Can Be Implemented in a Hospital Electronic Medical Record to Effectively Predict Inpatient Deterioration

Abstract

Objectives: To determine whether a statistically derived, trend-based, deterioration index is superior to other early warning scores at predicting adverse events and whether it can be integrated into an electronic medical record to enable real-time alerts.

Design: Forty-three variables and their trends from cases and controls were used to develop a logistic model and deterioration index to predict patient deterioration greater than or equal to 1 hour prior to an adverse event.

Setting: Two large Australian teaching hospitals.

Patients: Cases were considered as patients who suffered adverse events (unexpected death, unplanned ICU transfer, urgent surgery, and rapid-response alert) between August 1, 2016, and April 1, 2019.

Interventions: The logistic model and deterioration index were tested on historical data and then integrated into an electronic medical record for a 6-month prospective "silent" validation.

Measurements and main results: Data were acquired from 258,732 admissions. There were 8,002 adverse events. The addition of vital sign and laboratory trend values to the logistic model increased the area under the curve from 0.84 to 0.89 and the sensitivity to predict an adverse event 1-48 hours prior from 0.35 to 0.41. A 48-hour simulation showed that the logistic model had a higher area under the curve than the Modified Early Warning Score and National Early Warning Score (0.87 vs 0.74 vs 0.71). During the silently run prospective trial, the sensitivity of the deterioration index to detect adverse event any time prior to the adverse event was 0.474, 0.369 1 hour prior, and 0.327 4 hours prior, with a specificity of 0.972.

Conclusions: A deterioration prediction model was developed using patient demographics, ward-based observations, laboratory values, and their trends. The model's outputs were converted to a deterioration index that was successfully integrated into a live hospital electronic medical record. The sensitivity and specificity of the tool to detect inpatient deterioration were superior to traditional early warning scores.

Figure 1.

The receiver operating characteristic curve comparing the logistic model to the Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS) scores in the 48 hr prior to an adverse event. Simulations were run on a holdout sample of 86,989 patients. AUC = area under the curve.

Figure 2.

Deterioration index versus positive predictive value, true positive, and false positive alerts for the 48 hr prior to an adverse event. The simulation was performed on a holdout set of 86,989 admissions.