Administrative Data Are Unreliable for Ranking Hospital Performance Based on Serious Complications After Spine Fusion

Abstract

Study design: Retrospective analysis of administrative billing data.

Objective: To evaluate the extent to which a metric of serious complications determined from administrative data can reliably profile hospital performance in spine fusion surgery.

Summary of background data: While payers are increasingly focused on implementing pay-for-performance measures, quality metrics must reliably reflect true differences in performance among the hospitals profiled.

Methods: We used State Inpatient Databases from nine states to characterize serious complications after elective cervical and thoracolumbar fusion. Hierarchical logistic regression was used to risk-adjust differences in case mix, along with variability from low case volumes. The reliability of this risk-stratified complication rate (RSCR) was assessed as the variation between hospitals that was not due to chance alone, calculated separately by fusion type and year. Finally, we estimated the proportion of hospitals that had sufficient case volumes to obtain reliable (>0.7) complication estimates.

Results: From 2010 to 2017 we identified 154,078 cervical and 213,133 thoracolumbar fusion surgeries. 4.2% of cervical fusion patients had a serious complication, and the median RSCR increased from 4.2% in 2010 to 5.5% in 2017. The reliability of the RSCR for cervical fusion was poor and varied substantially by year (range 0.04-0.28). Overall, 7.7% of thoracolumbar fusion patients experienced a serious complication, and the RSCR varied from 6.8% to 8.0% during the study period. Although still modest, the RSCR reliability was higher for thoracolumbar fusion (range 0.16-0.43). Depending on the study year, 0% to 4.5% of hospitals had sufficient cervical fusion case volume to report reliable (>0.7) estimates, whereas 15% to 36% of hospitals reached this threshold for thoracolumbar fusion.

Conclusion: A metric of serious complications was unreliable for benchmarking cervical fusion outcomes and only modestly reliable for thoracolumbar fusion. When assessed using administrative datasets, these measures appear inappropriate for high-stakes applications, such as public reporting or pay-for-performance.Level of Evidence: 3.

Figure 1:

The risk-stratified complication rate (RSCR) and interquartile range for cervical (A) and thoracolumbar (B) fusion surgeries. Results are stratified by hospital mean annual case-volume, divided into three groups of approximately equal size. The x-axis labels show the mean annual volume cutoffs for each group in parentheses.

Figure 2:

The rankability of risk-stratified complication rate (RSCR) by year for cervical and thoracolumbar fusion surgeries.

Figure 3:

A heatmap showing the degree of consistency in hospital decile rank in RSCR by year for cervical (A-B) and thoracolumbar (C-D) fusion, separated into low (A & C) versus high volume (B & D) by tertiles of case-volume. Shading reflects hospital decile rank in each year, where a value of 1 indicates the lowest (best) decile in terms of RSCR, and 10 indicates the decile with the highest (worst) RSCR. Hospital vertical position was set based on RSCR rank in 2010 and held constant throughout the figure. Hospitals missing data for any year from 2010-2016 were excluded, explaining the smaller number of hospitals included for low volume compared to high volume heatmaps.