Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Feb;40(1):111-127.
doi: 10.1007/s10935-018-00534-z.

Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

Cady Berkel  1   2 , Carlos G Gallo  3 , Irwin N Sandler  4 , Anne M Mauricio  4 , Justin D Smith  3 , C Hendricks Brown  3
Affiliations

Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

Cady Berkel et al. J Prim Prev. 2019 Feb.

Abstract

The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the "New Beginnings Program" serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.

Keywords: Evidence-based programs; Implementation; Measurement; Parenting; Pragmatic measures; Technology.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest. Sandler is a developer of the NBP and has an LLC that trains facilitators to deliver the program. Remaining authors declare that they have no conflict of interest.

Conflict of Interest

The authors declare they have no conflicts of interest.

Figures

Figure 1.
Figure 1.
Partnership to Promote Sustainable Implementation of a Parenting Program for Divorcing Families
Figure 2.
Figure 2.
Theoretical Implementation Cascade Model for Evidence-Based Parenting Programs
Figure 3.
Figure 3.
Dynamic Implementation Monitoring and Feedback System Using Empirically Supported Indicators of Implementation
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Aarons GA, & Sawitzky AC (2006). Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 33, 3, 289–301. doi:10.1007/s10488-006-0039-1 - DOI - PMC - PubMed
    1. Atkins DC, Steyvers M, Imel ZE, & Smyth P (2014). Scaling up the evaluation of psychotherapy: Evaluating motivational interviewing fidelity via statistical text classification. Implementation Science, 9, 49, 1–11. doi:10.1186/1748-5908-9-49 - DOI - PMC - PubMed
    1. Berkel C, Mauricio AM, Sandler IN, Wolchik SA, Gallo CG, & Brown CH (2018). The cascading effects of multiple dimensions of implementation on program outcomes: A test of a theoretical model. Prevention Science, 19, 6, 782–94. doi:10.1007/s11121-017-0855-4 - DOI - PMC - PubMed
    1. Berkel C, Mauricio AM, Schoenfelder E, & Sandler IN (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12, 1, 23–33. doi:10.1007/s11121-010-0186-1 - DOI - PMC - PubMed
    1. Berkel C, Sandler IN, Wolchik SA, Brown CH, Gallo CG, Chiapa A, Mauricio AM, & Jones S (2018). “Home practice is the program:” Parents’ practice of program skills as predictors of outcomes in the New Beginnings Program effectiveness trial. Prevention Science, 19, 5, 663–73. doi:10.1007/s11121-016-0738-0 - DOI - PMC - PubMed

Publication types