How to evaluate a multi-country implementation-focused network: Reflections from the Quality of Care Network (QCN) evaluation

Abstract

Learning about how to evaluate implementation-focused networks is important as they become more commonly used. This research evaluated the emergence, legitimacy and effectiveness of a multi-country Quality of Care Network (QCN) aiming to improve maternal, newborn and child health (MNCH) outcomes. We examined the QCN global level, national and local level interfaces in four case study countries. This paper presents the evaluation team's reflections on this 3.5 year multi-country, multi-disciplinary project. Specifically, we examine our approach, methodological innovations, lessons learned and recommendations for conducting similar research. We used a reflective methodological approach to draw lessons on our practice while evaluating the QCN. A 'reflections' tool was developed to guide the process, which happened within a period of 2-4 weeks across the different countries. All country research teams held focused 'reflection' meetings to discuss questions in the tool before sharing responses with this paper's lead author. Similarly, the different lead authors of all eight QCN papers convened their writing teams to reflect on the process and share key highlights. These data were thematically analysed and are presented across key themes around the implementation experience including what went well, facilitators and critical methodological adaptations, what can be done better and recommendations for undertaking similar work. Success drivers included the team's global nature, spread across seven countries with members affiliated to nine institutions. It was multi-level in expertise and seniority and highly multidisciplinary including experts in medicine, policy and health systems, implementation research, behavioural sciences and MNCH. Country Advisory Boards provided technical oversight and support. Despite complexities, the team effectively implemented the QCN evaluation. Strong leadership, partnership, communication and coordination were key; as were balancing standardization with in-country adaptation, co-production, flattening hierarchies among study team members and the iterative nature of data collection. Methodological adaptations included leveraging technology which became essential during COVID-19, clear division of roles and responsibilities, and embedding capacity building as both an evaluation process and outcome, and optimizing technology use for team cohesion and quality outputs.

Fig 1. QCN Evaluation team structure.