Examining the equivalence of fidelity over two generations of KEEP implementation: A preliminary analysis

Abstract

Problem: In order to obtain and maintain positive outcomes garnered from evidence-based practice (EBP) models, it is necessary to implement them effectively in "real world" settings, to continually monitor intervention fidelity to prevent drift, and to train new staff due to turnover. The fidelity monitoring processes that are commonly employed in research settings are labor intensive and probably unrealistic to employ in community agencies given the additional burden and cost that they represent over and above the cost of implementing the EBP. Efficient strategies for implementing fidelity monitoring and staff training procedures within the inner context of agency settings are needed to promote agency self-sufficiency and program sustainability.

Method: A cascading implementation model was used whereby agencies who achieved proficiency in KEEP, an EBP designed to prevent placement disruptions in foster and kinship child welfare homes, were trained to take on fidelity management roles to improve the likelihood of program sustainability. Agency staff were trained to self-monitor fidelity and to train internal staff to achieve model fidelity. A web-based system for conducting fidelity assessments and for onsite/internal and remote program quality monitoring was utilized.

Results: Scores on fidelity ratings from streamed observations of intervention sessions showed no differences for foster parents treated by first generation interventionists trained by model developers compared to a second generation of interventionists trained by the first generation.

Conclusion and relevance to child welfare: Development of the local intra-agency capacity to manage quality intervention delivery is an important feature of successful EBP implementation. Use of the cascading implementation model appears to support the development of methods for effective monitoring of fidelity of the KEEP intervention, for training new staff, and ultimately for the development of internal methods for maintaining program sustainability and effectiveness.

Keywords: Cascading dissemination; Equivalence design; Implementation fidelity; KEEP intervention.

Fig. 1

Cascaded dissemination model implemented as part of routine agency care.

Fig. 2

Facilitator adherence rating form.

Fig. 3

The obtained 90% C. I. of the difference between G1 and G2 falls within the equivalence margin of +/− 1.0 SD of the G1 mean.