A Mixed-Method Application of the Program Sustainability Assessment Tool to Evaluate the Sustainability of 4 Pediatric Asthma Care Coordination Programs

Abstract

Introduction: As part of a cross-site evaluation of the implementation of an evidence-based intervention for pediatric asthma care coordination into low-income communities, we sought to understand the factors that influenced the programs' expected sustainability of the programs after external funding ended.

Methods: We administered the Center for Public Health Systems Science's Program Sustainability Assessment Tool, a 40-item instrument assessing 8 domains of sustainability capacity, to 12 key informants across 4 program sites. We developed open-ended probes for each domain. We examined patterns in site-specific and overall domain scores, and coded qualitative data to identify challenges and strategies in each domain.

Results: Across sites, the domains of program evaluation (cross-site mean, 5.4 on a scale of 1-7) and program adaptation (mean, 5.2) had the highest ratings (indicating a strong finding during program evaluation) and funding stability had the lowest rating (mean, 2.7). Scores varied most across sites in the domains of strategic planning (SD, 0.9) and funding stability (SD, 0.9). Qualitative data revealed key challenges, including how implementation difficulties and externally led implementation can impede planning for sustainability. Program leaders discussed multiple strategies for enhancing capacity within each domain, including capitalizing on the interconnectedness of all domains, such as using evaluation and communication strategies to bolster internal political support throughout the implementation process.

Conclusion: Findings indicating weak and strong domains were consistent with previous findings of studies that used the Program Sustainability Assessment Tool. The addition of qualitative probes yielded detailed data describing capacity strengths, weaknesses, and strategies to increase the likelihood that programs are sustained.

Figure

Mean PSAT Sustainability Capacity Scores by Domain and Site in the Merck Childhood Asthma Network. Possible scores range from 1 to 7, with a higher number indicating greater strength. Scores were determined based on assessments of site leaders (2–4 per site) of 5 items per domain. Abbreviation: FQHC, federally qualified health center. [Table: see text]