Designing and implementing research on a statewide quality improvement initiative: the DIAMOND study and initiative

Abstract

Objective: To demonstrate a rigorous methodology that optimally balanced internal validity with generalizability to evaluate a statewide collaborative that implemented an evidence-based, collaborative care model for depression management in primary care.

Study design and setting: Several operational features of the DIAMOND (Depression Improvement Across Minnesota, Offering a New Direction) Initiative suggested that the DIAMOND Study use a staggered implementation design with repeated cross-sections of patients across clinical settings. A multilevel recruitment strategy elicited virtually complete study participation from the medical groups, clinics, and health plans that coordinated efforts to deliver and reimburse DIAMOND care. Patient identification capitalized on large health plan claims databases to rapidly identify the population of patients newly treated for depression in DIAMOND clinics.

Results: The staggered implementation design and multilevel recruitment strategy made it possible to evaluate DIAMOND by holding confounding factors constant and accurately identifying an intent-to-treat population of patients treated for depression without intruding on or requiring effort from their clinics.

Conclusions: Recruitment and data collection from health plans, medical groups and clinics, and patients ensured a representative, intent-to-treat sample of study-enrolled patients. Separating patient identification from care delivery reduced threats of selection bias and enabled comparisons between the treated population and study sample. A key challenge is that intent-to-treat patients may not be exposed to DIAMOND which dilutes the effect size but offers realistic expectations of the impact of quality improvement in a population of treated patients.

Figure 1

Patient recruitment period, clinic pre-implementation and shaded post-implementation periods by training sequence.