Evaluation of the NHS England evidence-based interventions programme: a difference-in-difference analysis

Abstract

Background: The NHS England evidence-based interventions programme (EBI), launched in April 2019, is a novel nationally led initiative to encourage disinvestment in low value care.

Method: We sought to evaluate the effectiveness of this policy by using a difference-in-difference approach to compare changes in volume between January 2016 and February 2020 in a treatment group of low value procedures against a control group unaffected by the EBI programme during our period of analysis but subsequently identified as candidates for disinvestment.

Results: We found only small differences between the treatment and control group after implementation, with reductions in volumes in the treatment group 0.10% (95% CI 0.09% to 0.11%) smaller than in the control group (equivalent to 16 low value procedures per month). During the month of implementation, reductions in volumes in the treatment group were 0.05% (95% CI 0.03% to 0.06%) smaller than in the control group (equivalent to 7 low value procedures). Using triple difference estimators, we found that reductions in volumes were 0.35% (95% CI 0.26% to 0.44%) larger in NHS hospitals than independent sector providers (equivalent to 47 low value procedures per month). We found no significant differences between clinical commissioning groups that did or did not volunteer to be part of a demonstrator community to trial EBI guidance, but found reductions in volume were 0.06% (95% CI 0.04% to 0.08%) larger in clinical commissioning groups that posted a deficit in the financial year 2018/19 before implementation (equivalent to 4 low value procedures per month).

Conclusions: Our analysis shows that the EBI programme did not accelerate disinvestment for procedures under its remit during our period of analysis. However, we find that financial and organisational factors may have had some influence on the degree of responsiveness to the EBI programme.

Keywords: financial incentives; health policy; quality improvement; surgery.

Figure 1

Trends in evidence-based interventions programme (EBI) treatment and control group procedures. *Treatment group includes all EBI phase I category 2 procedures with the exception of removal of benign skin lesions. Control group includes EBI phase II procedures including repair of minimally symptomatic inguinal hernia, surgical intervention for chronic rhinosinusitis, kidney stone surgery and surgical intervention for benign prostatic hypertrophy. The grey lines reflect the launch of the EBI programme in April 2019, and the publication of statutory guidance in November 2018. This figure was created by the coauthors of this manuscript.