Performance in the Medicare Shared Savings Program After Accounting for Nonrandom Exit: An Instrumental Variable Analysis

Abstract

Background: Accountable care organizations (ACOs) in the Medicare Shared Savings Program (MSSP) are associated with modest savings. However, prior research may overstate this effect if high-cost clinicians exit ACOs.

Objective: To evaluate the effect of the MSSP on spending and quality while accounting for clinicians' nonrandom exit.

Design: Similar to prior MSSP analyses, this study compared MSSP ACO participants versus control beneficiaries using adjusted longitudinal models that accounted for secular trends, market factors, and beneficiary characteristics. To further account for selection effects, the share of nearby clinicians in the MSSP was used as an instrumental variable. Hip fracture served as a falsification outcome. The authors also tested for compositional changes among MSSP participants.

Setting: Fee-for-service Medicare, 2008 through 2014.

Patients: A 20% sample (97 204 192 beneficiary-quarters).

Measurements: Total spending, 4 quality indicators, and hospitalization for hip fracture.

Results: In adjusted longitudinal models, the MSSP was associated with spending reductions (change, -$118 [95% CI, -$151 to -$85] per beneficiary-quarter) and improvements in all 4 quality indicators. In instrumental variable models, the MSSP was not associated with spending (change, $5 [CI, -$51 to $62] per beneficiary-quarter) or quality. In falsification tests, the MSSP was associated with hip fracture in the adjusted model (-0.24 hospitalizations for hip fracture [CI, -0.32 to -0.16 hospitalizations] per 1000 beneficiary-quarters) but not in the instrumental variable model (0.05 hospitalizations [CI, -0.10 to 0.20 hospitalizations] per 1000 beneficiary-quarters). Compositional changes were driven by high-cost clinicians exiting ACOs: High-cost clinicians (99th percentile) had a 30.4% chance of exiting the MSSP, compared with a 13.8% chance among median-cost clinicians (50th percentile).

Limitation: The study used an observational design and administrative data.

Conclusion: After adjustment for clinicians' nonrandom exit, the MSSP was not associated with improvements in spending or quality. Selection effects-including exit of high-cost clinicians-may drive estimates of savings in the MSSP.

Primary funding source: Horowitz Foundation for Social Policy, Agency for Healthcare Research and Quality, and National Institute on Aging.

Figure 1.. Changes in Medicare spending for beneficiaries attributed to MSSP ACOs versus control beneficiaries

Mean outcomes for control beneficiaries were derived from the adjusted longitudinal model. Total spending was the sum of Medicare spending for inpatient, outpatient, professional, and skilled-nursing facility services. Component spending was defined by claims for services from the following research identifiable files: Medicare Provider Analysis and Review (for inpatient and skilled-nursing facility services); carrier (for professional services); and outpatient services (for outpatient services). ACO = accountable care organization; MSSP = Medicare Shared Savings Program.

Figure 2.. Changes in spending and hospitalization for hip fracture across models for fixed differences across MSSP participants vs. controls

Mean outcomes for control beneficiaries were derived from the adjusted longitudinal model. Hospitalization for hip fracture was defined as a Medicare acute-care hospital claim for a primary diagnosis of hip fracture (820.xx). Total spending was the sum of Medicare spending for inpatient, outpatient, professional, and skilled-nursing facility services. ACO = accountable care organization; MSSP = Medicare Shared Savings Program. Panel A. Change in rate of hospitalization for hip fracture per 1000 beneficiary-quarters. Panel B. Change in total spending per beneficiary-quarter.

Figure 3.. Changes in clinical quality performance for beneficiaries attributed to MSSP ACOs vs. controls

The proportions of MSSP beneficiaries and control beneficiaries receiving the preventive service are given in percent. Mean outcomes for control beneficiaries were derived from the adjusted longitudinal model. Diabetes clinical indicators were derived from National Quality Forum specifications and included glycated hemoglobin testing (in quarter of interest), LDL cholesterol testing (in quarter of interest or previous 3 quarters), and diabetic retinal examination (in quarter of interest or previous 3 quarters). Analyses were limited to beneficiaries with diabetes (n = 15 323 604 beneficiary-quarters). The mammography indicator was derived from National Quality Forum specifications. Beneficiaries were defined as meeting the indicator if they received a mammogram (in quarter of interest or previous 7 quarters). Mammography analyses were limited to female beneficiaries aged 65–69 y (n = 11 922 514 beneficiary-quarters). We excluded all observations following a mastectomy. ACO = accountable care organization; LDL = low-density lipoprotein; MSSP = Medicare Shared Savings Program.

Figure 4.. Association between clinician spending and probability of clinician exiting or entering the MSSP

Average clinician spending was defined as the average Medicare spending per beneficiary per year of the clinician's attributed patient panel in the 3 y before MSSP exit or entry determination. The probability of a clinician entering or exiting the MSSP was estimated as a function of the clinician's average spending, average beneficiary characteristics of the clinician's attributed patient panel, market fixed effects, and year fixed effects. Quadratic and cubic spending terms were included to allow for any potential nonlinearities in the effect of spending performance on MSSP participation. Analyses of MSSP exit (n = 21 418 clinician-years) were restricted to clinicians participating in the MSSP during the year before analysis. Analyses of MSSP entry (n = 161 957 clinician-years) were restricted to clinicians not participating in the MSSP during the year before analysis and did not include ACO formation, i.e., participation in an ACO's first contract year. Both sets of analyses were restricted to ACOs that entered MSSP contracts in 2012 or 2013, because 2012–2014 MSSP data could not be used to determine clinician exit or entry for ACOs formed in 2014. ACO = accountable care organization; MSSP = Medicare Shared Savings Program.