Can the American College of Surgeons NSQIP Surgical Risk Calculator Accurately Predict Adverse Postoperative Outcomes in Emergency Abdominal Surgery? An Italian Multicenter Analysis

Abstract

Background: The American College of Surgeons NSQIP surgical risk calculator provides an estimation of 30-day postoperative adverse outcomes. It is useful in the identification of high-risk patients needing clinical optimization and supports the informed consent process. The purpose of this study is to validate its predictive value in the Italian emergency setting.

Study design: Six Italian institutions were included. Inclusion diagnoses were acute cholecystitis, appendicitis, gastrointestinal perforation or obstruction. Areas under the receiving operating characteristic curves, Brier score, Hosmer-Lemeshow index, and observed-to-expected event ratio were measured to assess both discrimination and calibration. Effect of the Surgeon Adjustment Score on calibration was then tested. A patient's personal risk ratio was obtained, and a cutoff was chosen to predict mortality with a high negative predicted value.

Results: A total of 2,749 emergency procedures were considered for the analysis. The areas under the receiving operating characteristic curve were 0.932 for death (0.921 to 0.941, p < 0.0001; Brier 0.041) and 0.918 for discharge to nursing or rehabilitation facility (0.907 to 0.929, p < 0.0001; 0.070). Discrimination was also strong (area under the receiving operating characteristic curve >0.8) for renal failure, cardiac complication, pneumonia, venous thromboembolism, serious complication, and any complication. Brier score was informative (<0.25) for all the presented variables. The observed-to-expected event ratios were 1.0 for death and 0.8 for discharge to facility. For almost all other variables, there was a general risk underestimation, but the use of the Surgeon Adjustment Score permitted a better calibration of the model. A risk ratio >3.00 predicted the onset of death with sensitivity = 86%, specificity = 77%, and negative predicted value = 99%.

Conclusions: The American College of Surgeons NSQIP surgical risk calculator has proved to be a reliable predictor of adverse postoperative outcomes also in Italian emergency settings, with particular regard to mortality. We therefore recommend the use of the surgical risk calculator in the multidisciplinary care of patients undergoing emergency abdominal surgery.