Comparison of risk adjustment methodologies in surgical quality improvement

Abstract

Background: All hospitals are required to perform quality assurance activities. Many risk adjustment methodologies have been developed, and many medical centers use 1 or more than 1 risk adjustment program in an attempt to characterize their outcomes better rather than simply assessing unadjusted outcome statistics. The University HealthSystem Consortium (UHC) and American College of Surgeons-National Surgical Quality Improvement Program (NSQIP) both produce risk-adjusted outcome data. Our institution recognized a large disparity between our UHC and NSQIP risk-adjusted mortality. The purpose of this study was to attempt to discover the cause of that disparity.

Methods: One hundred twenty consecutive NSQIP records were matched with their UHC submissions during 2006. All patients' comorbidities and outcomes were reviewed, and the 2 systems, UHC and NSQIP, were compared for degree of discordance.

Results: Approximately twice the number of comorbidities per patient were documented in UHC (2.85+/-2.52) submissions compared with NSQIP (1.38+/-1.52, P < .001). The reporting of the comorbidities of hypertension, cardiac disease, pulmonary disease, and diabetes between UHC and NSQIP were similar in the percentage of patients reported as having each of those disease states, but the discordance between the 2 systems was 12%, 13%, 15%, and 5%, respectively (P < .001 in all 4). A total of 28% of patients were reported as suffering complications in NSQIP but only 11% in UHC, with a 26% rate of discordance (P < .01). Overall, 13% of patients were reported as having a surgical site infection in NSQIP, but only 1% in UHC.

Conclusions: We found significant differences in the reporting of both comorbidities and outcomes between our medical center's submissions to UHC and NSQIP in a consecutive series of patients. This may be at least partially responsible for the difference in the risk-adjusted mortality for our institution, as reported by UHC and NSQIP.