Assessment of six mortality prediction models in patients admitted with severe sepsis and septic shock to the intensive care unit: a prospective cohort study

Abstract

Introduction: We conducted the present study to assess the validity of mortality prediction systems in patients admitted to the intensive care unit (ICU) with severe sepsis and septic shock. We included Acute Physiology and Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, Mortality Probability Model (MPM) II0 and MPM II24 in our evaluation. In addition, SAPS II and MPM II24 were customized for septic patients in a previous study, and the customized versions were included in this evaluation.

Materials and method: This cohort, prospective, observational study was conducted in a tertiary care medical/surgical ICU. Consecutive patients meeting the diagnostic criteria for severe sepsis and septic shock during the first 24 hours of ICU admission between March 1999 and August 2001 were included. The data necessary for mortality prediction were collected prospectively as part of the ongoing ICU database. Predicted and actual mortality rates, and standardized mortality ratio were calculated. Calibration was assessed using Lemeshow-Hosmer goodness of fit C-statistic. Discrimination was assessed using receiver operating characteristic curves.

Results: The overall mortality prediction was adequate for all six systems because none of the standardized mortality ratios differed significantly from 1. Calibration was inadequate for APACHE II, SAPS II, MPM II0 and MPM II24. However, the customized version of SAPS II exhibited significantly improved calibration (C-statistic for SAPS II 23.6 [P = 0.003] and for customized SAPS II 11.5 [P = 0.18]). Discrimination was best for customized MPM II24 (area under the receiver operating characteristic curve 0.826), followed by MPM II24 and customized SAPS II.

Conclusion: Although general ICU mortality system models had accurate overall mortality prediction, they had poor calibration. Customization of SAPS II and, to a lesser extent, MPM II24 improved calibration. The customized model may be a useful tool when evaluating outcomes in patients with sepsis.

Figure 1

Receiver operating characteristic (ROC) curves for the six mortality prediction systems. APACHE, Acute Physiology and Chronic Health Evaluation; cus, customized; MPM, Mortality Probability Model; SAPS, Simplified Acute Physiology Score.