Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics

Abstract

Background: Collaborative-care management is an evidence-based practice for improving depression outcomes in primary care. The Department of Veterans Affairs (VA) has mandated the implementation of collaborative-care management in its satellite clinics, known as Community Based Outpatient Clinics (CBOCs). However, the organizational characteristics of CBOCs present added challenges to implementation. The objective of this study was to evaluate the effectiveness of evidence-based quality improvement (EBQI) as a strategy to facilitate the adoption of collaborative-care management in CBOCs.

Methods: This nonrandomized, small-scale, multisite evaluation of EBQI was conducted at three VA Medical Centers and 11 of their affiliated CBOCs. The Plan phase of the EBQI process involved the localized tailoring of the collaborative-care management program to each CBOC. Researchers ensured that the adaptations were evidence based. Clinical and administrative staff were responsible for adapting the collaborative-care management program for local needs, priorities, preferences and resources. Plan-Do-Study-Act cycles were used to refine the program over time. The evaluation was based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) Framework and used data from multiple sources: administrative records, web-based decision-support systems, surveys, and key-informant interviews.

Results: Adoption: 69.0% (58/84) of primary care providers referred patients to the program. Reach: 9.0% (298/3,296) of primary care patients diagnosed with depression who were not already receiving specialty care were enrolled in the program. Fidelity: During baseline care manager encounters, education/activation was provided to 100% (298/298) of patients, barriers were assessed and addressed for 100% (298/298) of patients, and depression severity was monitored for 100% (298/298) of patients. Less than half (42.5%, 681/1603) of follow-up encounters during the acute stage were completed within the timeframe specified. During the acute phase of treatment for all trials, the Patient Health Questionnaire (PHQ9) symptom-monitoring tool was used at 100% (681/681) of completed follow-up encounters, and self-management goals were discussed during 15.3% (104/681) of completed follow-up encounters. During the acute phase of treatment for pharmacotherapy and combination trials, medication adherence was assessed at 99.1% (575/580) of completed follow-up encounters, and side effects were assessed at 92.4% (536/580) of completed follow-up encounters. During the acute phase of treatment for psychotherapy and combination trials, counseling session adherence was assessed at 83.3% (239/287) of completed follow-up encounters. Effectiveness: 18.8% (56/298) of enrolled patients remitted (symptom free) and another 22.1% (66/298) responded to treatment (50% reduction in symptom severity). Maintenance: 91.9% (10/11) of the CBOCs chose to sustain the program after research funds were withdrawn.

Conclusions: Provider adoption was good, although reach into the target population was relatively low. Fidelity and maintenance were excellent, and clinical outcomes were comparable to those in randomized controlled trials. Despite the organizational barriers, these findings suggest that EBQI is an effective facilitation strategy for CBOCs.

Trial registration: Clinical trial # NCT00317018.

Figure 1

Percentage of providers referring to the depression care manager. To measure adoption, data were extracted from the Medical SAS Datasets at the Austin Information Technology Center. The post-period was defined as the 12 months after each CBOC start date (defined as the date the first patient was enrolled in the telemedicine-based CCM program), which ranged from April 2006 to February 2008. The number of primary care providers at each CBOC was determined from the SAS Medical Datasets using unique provider IDs for all types of primary care providers (e.g., general internist, advance practice nurse). The number of primary care providers referring a patient to a depression care manager during the 12-month post-period was identified from NetDSS. The adoption rate for each CBOC was defined as the total number of primary care providers referring a patient to the depression care manager (identified from NetDSS) during the 12-month period divided by the total number of primary care providers seeing patients during the 12-month period (identified from SAS Medical Datasets).

Figure 2

Percentage of patients referred to the depression care manager. To measure reach, data were extracted from the SAS Medical Datasets at the Austin Information Technology Center for the 12-month post-period. Index visits for patients during the 12-month post-period were defined as the first primary care encounter at the CBOC with a depression diagnosis. Patients were excluded if they had a specialty mental health visit or a diagnosis of bipolar disorder or schizophrenia during the six months prior to the CBOC start date or if the index visit was a specialty mental health encounter. The number of patients referred to the depression care manager during the 12-month post-period was identified from NetDSS. Reach during the 12-month post-period at each CBOC was defined as the total number of patients referred to the depression care manager (identified from NetDSS) divided by the total number of patients with a depression diagnosis who were not already receiving specialty care (identified in the SAS Medical Datasets).

Figure 3

Level of use of telemedicine-based collaborative-care management. The Level of Use interview measured sustained use of the program and was administered approximately a year after the last CBOC enrolled its first patient into the telemedicine-based CCM program, when research funds were no longer supporting the salary of clinical personnel. Level of Use was measured using key informant interviews with the Medical Directors of each of the 11 CBOCs and the Chief of Mental Health or Chief of Primary Care at the VAMC (depending on which service line operated the program). Using a structured interview guide and inductive questioning, the Level of Use framework classified the CBOCs into eight ranked levels of adoption according to their adoption behaviors. The first three levels distinguish between stages of nonuse (nonuse, orientation, and preparation). The next five levels distinguish between stages of use (mechanical, routine, refinement, integration, renewal), and these distinctions are made based on the type of adaptations or refinements that are being made to the innovation.

Figure 4

Level of institutionalization of telemedicine-based collaborative care management. Level of Institutionalization survey measured the degree to which the program was institutionalized within the organization. Level of Institutionalization was measured via telephone survey of the Chief of Mental Health or Chief of Primary Care at the VAMC (depending on which service line operated the program). Institutionalization implies that the organization has modified itself to incorporate the innovation and that the innovation has ceased to become novel and has been embedded in standard operating procedures. The Level of Institutionalization instrument measures an innovation's institutionalization among four subsystems: production, maintenance, supportive, and managerial. The production subsystem is responsible for delivering clinical services; to be institutionalized, the innovation must be integrated with other routine clinical services. The maintenance subsystem represents personnel; to be institutionalized, the innovation must be supported by permanent employees. The supportive subsystem represents external organizational forces; to be institutionalized, the innovation must have a stable source of funding and permanent office space. The managerial subsystem represents the executive and supervisory functions; to be institutionalized, the innovation must be assigned to a specific service, staff must have written job descriptions, and performance measures and progress reports must be required. For each subsystem, the Level of Institutionalization survey asks the respondent about the degree to which the organization has institutionalized the innovation (e.g., supported by permanent employees), and the responses are averaged to calculate an overall mean for each subsystem. The Level of Institutionalization instrument has three levels for each subsystem: low institutionalization (mean score ≤ 2), moderate institutionalization (2 < mean score ≤ 3) and high institutionalization (mean score > 3).