Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit

Abstract

Background: The Acute Physiology and Chronic Health Evaluation (APACHE) IV model has not yet been validated in Korea. The aim of this study was to compare the ability of the APACHE IV with those of APACHE II, Simplified Acute Physiology Score (SAPS) 3, and Korean SAPS 3 in predicting hospital mortality in a surgical intensive care unit (SICU) population.

Methods: We retrospectively reviewed electronic medical records for patients admitted to the SICU from March 2011 to February 2012 in a university hospital. Measurements of discrimination and calibration were performed using the area under the receiver operating characteristic curve (AUC) and the Hosmer-Lemeshow test, respectively. We calculated the standardized mortality ratio (SMR, actual mortality predicted mortality) for the four models.

Results: The study included 1,314 patients. The hospital mortality rate was 3.3%. The discriminative powers of all models were similar and very reliable. The AUCs were 0.80 for APACHE IV, 0.85 for APACHE II, 0.86 for SAPS 3, and 0.86 for Korean SAPS 3. Hosmer and Lemeshow C and H statistics showed poor calibration for all of the models (P < 0.05). The SMRs of APACHE IV, APACHE II, SAPS 3, and Korean SAPS 3 were 0.21, 0.11 0.23, 0.34, and 0.25, respectively.

Conclusions: The APACHE IV revealed good discrimination but poor calibration. The overall discrimination and calibration of APACHE IV were similar to those of APACHE II, SAPS 3, and Korean SAPS 3 in this study. A high level of customization is required to improve calibration in this study setting.

Keywords: Acute physiology and Chronic Health Evaluation II; Acute physiology and Chronic Health Evaluation IV; Intensive care unit; Simplified Acute Physiology Score 3; Validation.

Fig. 1

Study population. ICU: intensive care unit.

Fig. 2

Calibration plots of four different models (H-statistics). X-axis represents a decile predicted mortality of 10 groups on each model. The diagonal line indicates ideal prediction for hospital mortality. The bars represent the number of patients. APACHE: acute physiology and chronic health evaluation, SAPS: simplified acute physiology score.

Fig. 3

Calibration plots of four different models (C-statistics). X-axis represents an approximate decile patient of 10 groups. The bars represent the number of patients. The line with open circles represents mean predicted mortality. The line with closed circles represents mean observed mortality. APACHE: acute physiology and chronic health evaluation, SAPS: simplified acute physiology score.

Fig. 4

Comparison of the area under the receiver operating characteristic curves (AUC) of APACHE IV, APACHE II, SAPS 3, and Korean SAPS 3. The AUCs are 0.80, 0.85, 0.86, and 0.86 in APACHE IV, APACHE II, SAPS 3, Korean SAPS3 models, respectively. APACHE: acute physiology and chronic health evaluation, SAPS: simplified acute physiology score.