Prognostic performance of the Simplified Acute Physiology Score II in major Croatian hospitals: a prospective multicenter study

Abstract

Aim: To perform an external validation of the original Simplified Acute Physiology Score II (SAPS II) system and to assess its performance in a selected group of patients in major Croatian hospitals.

Methods: A prospective, multicenter study was conducted in five university hospitals and one general hospital during a six-month period between November 1, 2007 and May 1, 2008. Standardized hospital mortality ratio (SMR) was calculated from the mean predicted mortality of all the 2756 patients and the actual mortality for the same group of patients. The validation of SAPS II was made using the area under receiver operating characteristic curve (AUC), 2×2 classification tables, and Hosmer-Lemeshow tests.

Results: The predicted mortality was as low as 14.6% due to a small proportion of medical patients and the SMR being 0.89 (95% confidence interval [CI], 0.78-0.98). The SAPS II system demonstrated a good discriminatory power as measured by the AUC (0.85; standard error [SE]=0.012; 95% CI=0.840-0.866; P<0.001). This system significantly overestimated the actual mortality (Hosmer-Lemeshow goodness-of-fit H statistic: χ(2) =584.4; P<0.001 and C statistics: χ(2)(8) =313.0; P<0.001) in the group of patients included in the study.

Conclusion: The SAPS II had a good discrimination, but it significantly overestimated the observed mortality in comparison with the predicted mortality in this group of patients in Croatia. Therefore, caution is required when an evaluation is performed at the individual level.

Figure 1

Receiver operating characteristic (ROC) curve for the Simplified Acute Physiology Score II (curved line connected with the circles) including standard deviation lines (curved dotted lines) in intensive care units (ICU) in major Croatian hospitals and example of ROC value of 0.5 (diagonal line). The relationship between true positives (sensitivity) and false positives (100-specificity) is presented. The area under the ROC curve was 0.85.

Figure 2

Calibration curve for the Simplified Acute Physiology Score II. The solid line represents a perfect correspondence between the observed and predicted mortality; the dotted line represents the observed vs predicted mortality; columns show the distribution of the patients in the analyzed groups.