Estimates of ACO savings in the presence of provider and beneficiary selection

Abstract

Background: Medicare's accountable care organizations (ACOs)-designed to improve quality and lower spending-were associated with growing savings in previous studies. However, savings estimates may be biased by beneficiary sorting among providers based on healthcare needs and by providers opting into the program based on anticipated gains.

Methods: Using Medicare administrative claims (2009-2014), we compared annual spending changes after provider organizations joined ACOs to changes in non-ACOs (controls). To address provider selection, using novel data to identify non-ACO organizations, we restricted controls to comparably large provider organizations. To address beneficiary selection, we (a) estimated within-organization (including non-ACO comparison organizations) spending changes, (b) estimated within-beneficiary spending changes, (c) incorporated beneficiaries without qualifying healthcare expenses, and (d) used a fixed beneficiary ACO assignment using the pre-ACO period.

Results: Each year, 19% of Medicare beneficiaries switched provider organizations. Spending was higher for switchers than stayers ($3163, p < .001) and grew more the next year ($2004; p < .001). Starting from a baseline regression modeled on previous ACO evaluations, estimated savings varied widely as we sequentially introduced methods to address selection. Combining methods, however, generated more stable estimated ACO savings of $46 (p = .022), averaged across cohorts.

Conclusions: When implementing a comprehensive suite of methods to adjust for provider and beneficiary selection, we estimated ACO savings that grew over time. Our estimates are in line with, but smaller than, previous estimates in the literature. Implementing piecemeal adjustments produced misleading results.

Implications: Our results confirm the importance of selection for savings estimates and for provider organizations managing costs and quality. Attribution rules that consider multiple years may help mitigate the impact of beneficiary churn for providers and payers. Implementing payment reform by randomizing early participants, or implementing fully across selected markets, may better serve efforts to evaluate and improve payment models.

Level of evidence: Level 3.

Keywords: Accountable Care Organization (ACO); Alternative payment models; Evaluation methodology; Medicare; Selection; Shared Savings Program.

Figure 1. Trends in annual spending per beneficiary

Notes: Dashed black lines represent non-ACOs. Solid colored lines represent organizations that join ACOs in post period: Pioneer, 2012 MSSP, 2013 MSSP, and 2014 MSSP in order from lightest to darkest. Vertical line divides pre and post periods.

Figure 2. Comparing pooled estimates of ACO effect across regressions

Notes: Plot of results from regressions identical to those in Table 4 except that a pooled ACO effect is estimated rather than a separate effect by cohort and participation year. Estimates shown as dots and 95% confidence intervals as bars.