Benchmarking trauma centers on mortality alone does not reflect quality of care: implications for pay-for-performance

Abstract

Background: Trauma centers are currently benchmarked on mortality outcomes alone. However, pay-for-performance measures may financially penalize centers based on complications. Our objective was to determine whether the results would be similar to the current standard method of mortality-based benchmarking if trauma centers were profiled on complications.

Methods: We analyzed data from the National Trauma Data Bank from 2007 to 2010. Patients 16 years or older with blunt or penetrating injuries and an Injury Severity Score (ISS) of 9 or higher were included. Risk-adjusted observed-to-expected (O/E) mortality ratios for each center were generated and used to rank each facility as high, average, or low performing. We similarly ranked facilities on O/E morbidity ratios defined as occurrence of any major complication. Concordance between hospital performance rankings was evaluated using a weighted κ statistic. Correlation between morbidity- and mortality-based O/E ratios was assessed using Pearson coefficients. Sensitivity analyses were performed to mitigate the competing risk of death for the morbidity analyses.

Results: A total of 449,743 patients from 248 facilities were analyzed. The unadjusted morbidity and mortality rates were 10.0% and 6.9%, respectively. No correlation was found between morbidity- and mortality-based O/E ratios (r = -0.01). Only 40% of the centers had similar performance rankings for both mortality and morbidity. Of the 31 high performers for mortality, only 11 centers were also high performers for morbidity. A total of 78 centers were ranked as average, and 11 ranked as low performers on both outcomes. Comparison of hospital performance status using mortality and morbidity outcomes demonstrated poor concordance (weighted κ = 0.03, p = 0.22).

Conclusion: Mortality-based external benchmarking does not identify centers with high complication rates. This creates a dichotomy between current trauma center profiling standards and measures used for pay-for-performance. A benchmarking mechanism that reflects all measures of quality is needed.

Level of evidence: Prognostic/epidemiologic study, level III.