Which elements of improvement collaboratives are most effective? A cluster-randomized trial

Abstract

Aims: Improvement collaboratives consisting of various components are used throughout health care to improve quality, but no study has identified which components work best. This study tested the effectiveness of different components in addiction treatment services, hypothesizing that a combination of all components would be most effective.

Design: An unblinded cluster-randomized trial assigned clinics to one of four groups: interest circle calls (group teleconferences), clinic-level coaching, learning sessions (large face-to-face meetings) and a combination of all three. Interest circle calls functioned as a minimal intervention comparison group.

Setting: Out-patient addiction treatment clinics in the United States.

Participants: Two hundred and one clinics in five states.

Measurements: Clinic data managers submitted data on three primary outcomes: waiting-time (mean days between first contact and first treatment), retention (percentage of patients retained from first to fourth treatment session) and annual number of new patients. State and group costs were collected for a cost-effectiveness analysis.

Findings: Waiting-time declined significantly for three groups: coaching (an average of 4.6 days/clinic, P = 0.001), learning sessions (3.5 days/clinic, P = 0.012) and the combination (4.7 days/clinic, P = 0.001). The coaching and combination groups increased significantly the number of new patients (19.5%, P = 0.028; 8.9%, P = 0.029; respectively). Interest circle calls showed no significant effect on outcomes. None of the groups improved retention significantly. The estimated cost per clinic was $2878 for coaching versus $7930 for the combination. Coaching and the combination of collaborative components were about equally effective in achieving study aims, but coaching was substantially more cost-effective.

Conclusions: When trying to improve the effectiveness of addiction treatment services, clinic-level coaching appears to help improve waiting-time and number of new patients while other components of improvement collaboratives (interest circles calls and learning sessions) do not seem to add further value.

Figure 1

NIATx 200 Study Design

Figure 2. Flow diagram

^a “Lost to follow up” indicates clinics that did not send primary outcomes data to state data managers. Clinics that did send in primary outcomes data were included in the intent-to-treat analysis. Cleaning scripts were run on the data sent by the state data managers to the research team, and clean data were entered into a Microsoft SQL Server database. Loss to follow-up rates did not differ significantly between groups: interest circle calls (8.2%), coaching (6.0%), learning sessions (5.6%), and combination (0.0%). ^b Most clinics that were lost to follow up had scheduling conflicts that precluded their participating in QI activities. Most clinic staff have only limited control over their schedules. ^c “Hours” are hours of participation per clinic, unweighted by the number of clinic staff members participating from each clinic. The interest circle call group, with 18 maximum possible hours of participation, was conceived as a type of control or comparison group.