Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (ADEPT): approaches to adopting implementation strategies

Abstract

Background: Theory-based methods to support the uptake of evidence-based practices (EBPs) are critical to improving mental health outcomes. Implementation strategy costs can be substantial, and few have been rigorously evaluated. The purpose of this study is to conduct a cost-effectiveness analysis to identify the most cost-effective approach to deploying implementation strategies to enhance the uptake of Life Goals, a mental health EBP.

Methods: We used data from a previously conducted randomized trial to compare the cost-effectiveness of Replicating Effective Programs (REP) combined with external and/or internal facilitation among sites non-responsive to REP. REP is a low-level strategy that includes EBP packaging, training, and technical assistance. External facilitation (EF) involves external expert support, and internal facilitation (IF) augments EF with protected time for internal staff to support EBP implementation. We developed a decision tree to assess 1-year costs and outcomes for four implementation strategies: (1) REP only, (2) REP+EF, (3) REP+EF add IF if needed, (4) REP+EF/IF. The analysis used a 1-year time horizon and assumed a health payer perspective. Our outcome was quality-adjusted life years (QALYs). The economic outcome was the incremental cost-effectiveness ratio (ICER). We conducted deterministic and probabilistic sensitivity analysis (PSA).

Results: Our results indicate that REP+EF add IF is the most cost-effective option with an ICER of $593/QALY. The REP+EF/IF and REP+EF only conditions are dominated (i.e., more expensive and less effective than comparators). One-way sensitivity analyses indicate that results are sensitive to utilities for REP+EF and REP+EF add IF. The PSA results indicate that REP+EF, add IF is the optimal strategy in 30% of iterations at the threshold of $100,000/QALY.

Conclusions: Our results suggest that the most cost-effective implementation support begins with a less intensive, less costly strategy initially and increases as needed to enhance EBP uptake. Using this approach, implementation support resources can be judiciously allocated to those clinics that would most benefit. Our results were not robust to changes in the utility measure. Research is needed that incorporates robust and relevant utilities in implementation studies to determine the most cost-effective strategies. This study advances economic evaluation of implementation by assessing costs and utilities across multiple implementation strategy combinations.

Trial registration: ClinicalTrials.gov Identifier: NCT02151331 , 05/30/2014.

Keywords: Cost-effectiveness analysis; Costs and cost analysis; Implementation science; Mental health services, community.

Fig. 1

Decision tree of the ADEPT trial. ^aSites that responded to the implementation strategy after the initial 6 months of the Trial Phase: either < 10 patients receiving Life Goals or > 50% of patients receiving Life Goals had ≤ 3 sessions, min dose for clinically significant results. Sites that responded to the implementation strategy discontinued the strategy during the second 6 months/Phase III of the trial

Fig. 2

Cost-effectiveness plane, organization/payer perspective

Fig. 3

Tornado diagram showing one-way sensitivity analyses for the base case with the most sensitive parameters. All parameters were evaluated and data are provided in the appendix. Thick vertical black lines on the ends of the bars indicate values at which the initial preferred option is no longer cost-effective

Fig. 4

The original study design to evaluate effectiveness (a) and decision tree model to evaluate cost-effectiveness (b). This cost-effectiveness analysis focuses on implementation strategies for sites not responding to the REP alone intervention (the “sites not responding to REP alone” portion of the tree in 2a). In the original study, baseline data were gathered prior to initiation of the trial phase (Phase I). In this study, we sought to determine the most cost-effective option for deploying an implementation strategy with multiple components across its all possible permutations (e.g., REP+EF/IF) and comparing this to usual implementation (baseline REP). To accomplish this, we created the decision tree to represent all the decision options and their subsequent steps and estimate their respective costs and consequences to allow for comparison. This modeling approach represents the possible implementation strategy decision options for decision makers, quantifies the uncertainty, and allows for evaluation of alternatives. ^a In the original trial, non-responding sites were randomized following Phase I to REP+EF or REP+EF/IF. ^b Following Phase II, non-responding sites in the REP+EF condition were randomized again to either continue REP+EF or add IF (REP+EF/IF). Details of the trial are published elsewhere (see Kilbourne et. al., 2014). ^c Sites that responded to the implementation strategy after the initial 6 months of the Trial Phase: either < 10 patients receiving Life Goals or > 50% of patients receiving Life Goals had ≤ 3 sessions, min dose for clinically significant results. Sites that responded to the implementation strategy discontinued the strategy during the second 6 months/Phase III of the trial