Research and Evaluation in a Child-Focused Place-Based Initiative: West Philly Promise Neighborhood

Abstract

Place-based initiatives attempt to reduce persistent health inequities through multisectoral, cross-system collaborations incorporating multiple interventions targeted at varying levels from individuals to systems. Evaluations of these initiatives may be thought of as part of the community change process itself with a focus on real-time learning and accountability. We described the design, implementation, challenges, and initial results of an evaluation of the West Philly Promise Neighborhood, which is a comprehensive, child-focused place-based initiative in Philadelphia, Pennsylvania. Priorities for the evaluation were to build processes for and a culture of ongoing data collection, monitoring, and communication, with a focus on transparency, accountability, and data democratization; establish systems to collect data at multiple levels, with a focus on multiple uses of the data and future sustainability; and adhere to grant requirements on data collection and reporting. Data collection activities included the compilation of neighborhood-level indicators; the implementation of a program-tracking system; administrative data linkage; and neighborhood, school, and organizational surveys. Baseline results pointed to existing strengths in the neighborhood, such as the overwhelming majority of caregivers reporting that they read to their young children (86.9%), while other indicators showed areas of need for additional supports and were programmatic focuses for the initiative (e.g., about one-quarter of young children were not engaged in an early childhood education setting). Results were communicated in multiple formats. Challenges included aligning timelines, the measurement of relationship-building and other process-focused outcomes, data and technology limitations, and administrative and legal barriers. Evaluation approaches and funding models that acknowledge the importance of capacity-building processes and allow the development and measurement of population-level outcomes in a realistic timeframe are critical for measuring the success of place-based approaches.

Keywords: Promise Neighborhood; children; community-engaged research; data integration; multisectoral; place-based intervention; program evaluation; systems thinking.

Figure 1

West Philly Promise Neighborhood theory of change.

Figure 2

Evaluation approach. Data collection methods were designed to capture information at each of the relevant levels of influence. For example, community-level data were collected through community indicators, a population-based neighborhood resident survey, and administrative data linkage. Some data collection methods also provided information about multiple different levels of influence. Stakeholder input based on communication of data and research results is incorporated into future data collection and communication activities. ^a Grant requirement. ^b School survey, and associated online data reporting and requests, developed and maintained by School District.