Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*

Abstract

Objective: Over 200,000 in-hospital cardiac arrests occur in the United States each year and many of these events may be preventable. Current vital sign-based risk scores for ward patients have demonstrated limited accuracy, which leads to missed opportunities to identify those patients most likely to suffer cardiac arrest and inefficient resource utilization. We derived and validated a prediction model for cardiac arrest while treating ICU transfer as a competing risk using electronic health record data.

Design: A retrospective cohort study.

Setting: An academic medical center in the United States with approximately 500 inpatient beds.

Patients: Adult patients hospitalized from November 2008 until August 2011 who had documented ward vital signs.

Interventions: None.

Measurements and main results: Vital sign, demographic, location, and laboratory data were extracted from the electronic health record and investigated as potential predictor variables. A person-time multinomial logistic regression model was used to simultaneously predict cardiac arrest and ICU transfer. The prediction model was compared to the VitalPAC Early Warning Score using the area under the receiver operating characteristic curve and was validated using three-fold cross-validation. A total of 56,649 controls, 109 cardiac arrest patients, and 2,543 ICU transfers were included. The derived model more accurately detected cardiac arrest (area under the receiver operating characteristic curve, 0.88 vs 0.78; p < 0.001) and ICU transfer (area under the receiver operating characteristic curve, 0.77 vs 0.73; p < 0.001) than the VitalPAC Early Warning Score, and accuracy was similar with cross-validation. At a specificity of 93%, our model had a higher sensitivity than the VitalPAC Early Warning Score for cardiac arrest patients (65% vs 41%).

Conclusions: We developed and validated a prediction tool for ward patients that can simultaneously predict the risk of cardiac arrest and ICU transfer. Our model was more accurate than the VitalPAC Early Warning Score and could be implemented in the electronic health record to alert caregivers with real-time information regarding patient deterioration.

Figure 1

Empirical density plots of the cardiac arrest score for cardiac arrest and for patients who experienced neither a cardiac arrest nor an ICU transfer.

Figure 2

Lowess smoother curves of the average cardiac arrest score within 48 hours of cardiac arrest or ICU transfer. The average score over a random 48 hours is shown for controls.