Improving quality of stroke care through benchmarking center performance: why focusing on outcomes is not enough

Abstract

Background: Between-center variation in outcome may offer opportunities to identify variation in quality of care. By intervening on these quality differences, patient outcomes may be improved. However, whether observed differences in outcome reflect the true quality improvement potential is not known for many diseases. Therefore, we aimed to analyze the effect of differences in performance on structure and processes of care, and case-mix on between-center differences in outcome after endovascular treatment (EVT) for ischemic stroke.

Methods: In this observational cohort study, ischemic stroke patients who received EVT between 2014 and 2017 in all 17 Dutch EVT-centers were included. Primary outcome was the modified Rankin Scale, ranging from 0 (no symptoms) to 6 (death), at 90 days. We used random effect proportional odds regression modelling, to analyze the effect of differences in structure indicators (center volume and year of admission), process indicators (time to treatment and use of general anesthesia) and case-mix, by tracking changes in tau², which represents the amount of between-center variation in outcome.

Results: Three thousand two hundred seventy-nine patients were included. Performance on structure and process indicators varied significantly between EVT-centers (P < 0.001). Predicted probability of good functional outcome (modified Rankin Scale 0-2 at 90 days), which can be interpreted as an overall measure of a center's case-mix, varied significantly between 17 and 50% across centers. The amount of between-center variation (tau²) was estimated at 0.040 in a model only accounting for random variation. This estimate more than doubled after adding case-mix variables (tau²: 0.086) to the model, while a small amount of between-center variation was explained by variation in performance on structure and process indicators (tau²: 0.081 and 0.089, respectively). This indicates that variation in case-mix affects the differences in outcome to a much larger extent.

Conclusions: Between-center variation in outcome of ischemic stroke patients mostly reflects differences in case-mix, rather than differences in structure or process of care. Since the latter two capture the real quality improvement potential, these should be used as indicators for comparing center performance. Especially when a strong association exists between those indicators and outcome, as is the case for time to treatment in ischemic stroke.

Keywords: Benchmarking; Case-mix; Endovascular treatment; Outcome differences; Process of care; Quality of care; Stroke.

Fig. 1

Center volume in each intervention year. Center volume is defined as percentage of all EVT patients treated in each center relative to all EVT patients treated in the Netherlands in that year

Fig. 2

Forest plots reporting random center effect (odds ratios and 95% confidence intervals) on inverse of modified Rankin Scale at 90 days in four models using random effect proportional odds regression analysis. a: Model 1 (unadjusted model); b: Model 2 (case-mix adjusted model); c: Model 3 (case-mix and structure indicators adjusted model); d: Model 4 (case-mix, structure and process indicators adjusted model)