Opening the 'black box' of collaborative improvement: a qualitative evaluation of a pilot intervention to improve quality of malaria surveillance data in public health centres in Uganda

Abstract

Background: Demand for high-quality surveillance data for malaria, and other diseases, is greater than ever before. In Uganda, the primary source of malaria surveillance data is the Health Management Information System (HMIS). However, HMIS data may be incomplete, inaccurate or delayed. Collaborative improvement (CI) is a quality improvement intervention developed in high-income countries, which has been advocated for low-resource settings. In Kayunga, Uganda, a pilot study of CI was conducted in five public health centres, documenting a positive effect on the quality of HMIS and malaria surveillance data. A qualitative evaluation was conducted concurrently to investigate the mechanisms of effect and unintended consequences of the intervention, aiming to inform future implementation of CI.

Methods: The study intervention targeted health workers, including brief in-service training, plus CI with 'plan-do-study-act' (PDSA) cycles emphasizing self-reflection and group action, periodic learning sessions, and coaching from a CI mentor. Health workers collected data on standard HMIS out-patient registers. The qualitative evaluation (July 2015 to September 2016) included ethnographic observations at each health centre (over 12-14 weeks), in-depth interviews with health workers and stakeholders (n = 20), and focus group discussions with health workers (n = 6).

Results: The results suggest that the intervention did facilitate improvement in data quality, but through unexpected mechanisms. The CI intervention was implemented as planned, but the PDSA cycles were driven largely by the CI mentor, not the health workers. In this context, characterized by a rigid hierarchy within the health system of limited culture of self-reflection and inadequate training and supervision, CI became an effective form of high-quality training with frequent supervisory visits. Health workers appeared motivated to improve data collection habits by their loyalty to the CI mentor and the potential for economic benefits, rather than a desire for self-improvement.

Conclusions: CI is a promising method of quality improvement and could have a positive impact on malaria surveillance data. However, successful scale-up of CI in similar settings may require deployment of highly skilled mentors. Further research, focusing on the effectiveness of 'real world' mentors using robust study designs, will be required to determine whether CI can be translated effectively and sustainably to low-resource settings.

Keywords: Collaborative improvement; Data quality; Health Management Information System (HMIS); Malaria; Public sector; Qualitative; Quality improvement; Surveillance; Uganda.

Fig. 1

Map of the study area in Kayunga district, Uganda, including the location of private and public (government-run) health centres. HC: health centre; HCII: level II health centre; HCIII: level III health centre; HCIV: level IV health centre

Fig. 2

Trial schema, outlining the timelines and study activities, including the components of the intervention and evaluation. CI: collaborative improvement; PDSA: plan-do-study-act