Earned outcome metrics outperform recommended open abdominal aortic aneurysm repair volume guidelines in discriminating perioperative mortality among centers

Abstract

Objective: The Society for Vascular Surgery (SVS) has endorsed a minimum open abdominal aortic aneurysm repair (OAR) volume of ≥10/year as an indicator of high-quality care. However, this threshold excludes most United States centers, despite an acceptable average mortality rate nationwide. In sports metrics, "replacement level" is used as a baseline for comparison, representing below-average quality of a readily available replacement player. We hypothesized that a novel earned outcome metric, reported relative to the quality of a "replacement level" center, would outperform the SVS volume threshold while still adequately representing high OAR quality overall.

Methods: Elective infrarenal OAR from the Vascular Quality Initiative database were studied, from 2016 to 2019. Risk-adjusted surgical quality was estimated using earned outcomes methods to analyze in-hospital/30-day death: deaths above replacement (DAR) were calculated as the number of observed deaths relative to predicted deaths at low-volume "replacement level" centers (observed - expected). Centers were categorized according to: (1) case volume (≥10 OAR/year); and (2) quality (DAR <0). The association of volume and DAR categorizations with crude and risk-adjusted mortality were assessed. Concordance of center categorization using the two thresholds and the ability of 2016 to 2019 center characterization to forecast OAR mortality for 2020 to 2023 were estimated.

Results: In total, 3701 patients underwent OAR at 106 centers during the study period. Using the SVS volume threshold, 33 centers (31%) performed ≥10 OAR/year. Using the earned outcomes threshold, 74 centers (70%) had DAR <0. Slightly over one-half of centers were classified similarly using both criteria (56%). However, using the DAR threshold, 44 centers with <10 OAR/year (42%) had DAR <0. Although reduced crude mortality rates were seen among patients undergoing OAR at centers with DAR <0 (3.0% vs 11.4%; P < .001) and at centers with ≥10 OAR/year (3.4% vs 6.0%; P < .001), the absolute mortality difference was higher using DAR <0 (8.4%) compared with ≥10 OAR/year (2.5%). Differences in risk-adjusted mortality also demonstrated better discrimination of mortality at centers with DAR <0 (3.1% vs 10.7%) compared with OAR ≥10/year (3.5% vs 5.6%). Among centers classified concordantly using both thresholds, mortality rates were similar in the subsequent 4 years (2020-2023). However, for centers classified discordantly, subsequent mortality rates were consistent with initial DAR characterization, even when they did not meet the volume threshold.

Conclusions: Measuring OAR quality is critically important to patients, payers, and providers. An earned outcomes threshold in OAR outperforms the SVS volume threshold for identifying high-quality centers, largely due to detection of low-volume centers with above-average outcomes. Metrics that account for both quality and volume should be utilized when benchmarking performance.

Keywords: Abdominal aortic aneurysm repair; Benchmarking; Performance metrics; Quality metrics.