Outcomes of the ROSE Sustainment (ROSES) Study, a sequential multiple assignment randomized implementation trial to determine the minimum necessary intervention to sustain a postpartum depression prevention program in agencies serving low-income pregnant people

Abstract

Background: This Sequential Multiple Assignment Randomized Trial (SMART) was conducted to determine minimum implementation support needed for agencies serving pregnant people on public assistance to adopt and sustain the ROSE (Reach Out, Stay Strong, Essentials for mothers of newborns) postpartum depression (PPD) prevention program.

Methods: Enrolled prenatal agencies (N=98) received thorough initial implementation support (initial training + written sustainment planning). Agencies were identified as at risk for non-sustainment within the first 15 months (N=56) were randomized to: (1) no additional implementation support (N=12), or (2) quarterly implementation support (coaching and feedback; N=44). If agencies receiving quarterly implementation supports were still at risk and within the first 15 months (N=29), they were randomized to: (1) continued quarterly support (N=14), or (2) monthly implementation support (N=15). No implementation support occurred after 18 months. Follow-ups occurred quarterly and then at 18, 24, and 30 months. Outcomes included sustainment of core program elements, agency PPD rates, reach, and costs/cost-effectiveness of each sustainment step.

Results: Twice as many agencies as expected (41 of 98; 42%) delivered ROSE with fidelity for 15+ months after receiving thorough initial implementation support only. For agencies at risk for non-sustainment, no effects of adding quarterly implementation supports were observed. However, adding monthly supports (versus quarterly) for agencies still at risk resulted in higher monthly percent of core ROSE elements sustained and more months ROSE was sustained with fidelity with large (Cohen's d = 0.73 and 0.80) effect sizes, and improved reach over 30 months. Many agencies did not consistently collect PPD rates, making results difficult to interpret. Mean implementation costs (including implementation support and agency staff time) per agency were $1,849 (SD $1,429) for agencies receiving initial implementation support only, $2,699 (SD $1,837) for those receiving initial and quarterly implementation support, and $4,059 (SD $1,763) for those receiving initial, quarterly, and ultimately monthly implementation support.

Conclusions: The cost of agency-wide ROSE implementation is far less than the cost of a single untreated case of PPD ($33,484). We suggest implementing ROSE through thorough training and written sustainment planning. For agencies not sustaining, adding monthly support can promote sustainment and improve reach.

Trial registration: Registered June 14, 2018 at clinicaltrials.gov, NCT03267563 ( https://clinicaltrials.gov/study/NCT03267563 ).

Keywords: Cost-effectiveness; Implementation; Postpartum depression; Prenatal care; Prevention; Public assistance; Sustainment.

Fig. 1

Agency-level CONSORT Diagram

Fig. 2

Time periods in months during which ROSE was offered with adequate fidelity*. *Agencies were not randomized after a period of no ROSE if they: (1) had a ROSE session scheduled (N=6); (2) held a ROSE session but no one came (therefore, no fidelity; N=2); or (3) the period occurred during the study’s randomization pause from 3/15/20 to 7/31/20 for COVID (N=3). In two cases, agencies were randomized who were later found to have adequate fidelity; one agency sent fidelity forms >6 months after the end of the reporting period, and another agency had a facilitator who misunderstood the fidelity checklists.