How does the effectiveness of strategies to improve healthcare provider practices in low-income and middle-income countries change after implementation? Secondary analysis of a systematic review

Abstract

Background: A recent systematic review evaluated the effectiveness of strategies to improve healthcare provider (HCP) performance in low-income and middle-income countries. The review identified strategies with varying effects, including in-service training, supervision and group problem-solving. However, whether their effectiveness changed over time remained unclear. In particular, understanding whether effects decay over time is crucial to improve sustainability.

Methods: We conducted a secondary analysis of data from the aforementioned review to explore associations between time and effectiveness. We calculated effect sizes (defined as percentage-point (%-point) changes) for HCP practice outcomes (eg, percentage of patients correctly treated) at each follow-up time point after the strategy was implemented. We estimated the association between time and effectiveness using random-intercept linear regression models with time-specific effect sizes clustered within studies and adjusted for baseline performance.

Results: The primary analysis included 37 studies, and a sensitivity analysis included 77 additional studies. For training, every additional month of follow-up was associated with a 0.19 %-point decrease in effectiveness (95% CI: -0.36 to -0.03). For training combined with supervision, every additional month was associated with a 0.40 %-point decrease in effectiveness (95% CI: -0.68 to -0.12). Time trend results for supervision were inconclusive. For group problem-solving alone, time was positively associated with effectiveness, with a 0.50 %-point increase in effect per month (95% CI: 0.37 to 0.64). Group problem-solving combined with training was associated with large improvements, and its effect was not associated with time.

Conclusions: Time trends in the effectiveness of different strategies to improve HCP practices vary among strategies. Programmes relying solely on in-service training might need periodical refresher training or, better still, consider combining training with group problem-solving. Although more high-quality research is needed, these results, which are important for decision-makers as they choose which strategies to use, underscore the utility of studies with multiple post-implementation measurements so sustainability of the impact on HCP practices can be assessed.

Keywords: health services research; implementation science; performance measures; quality improvement; standards of care.

Figure 1

Effectiveness of five strategies over time: (A) training alone, (B) supervision alone, (C) training plus supervision, (D) group problem-solving alone, (E) group problem-solving plus training. Data from all studies with an interrupted time series (ITS) design or non-ITS design with at least two follow-up measurements are shown for each strategy. Each study’s data are represented by one colour, and effect sizes from the same outcome are connected with a line. The middle dashed line in the shaded area indicates the predicted effect size at each time point based on the random-intercept linear regression model with baseline healthcare provider performance held constant at the mean level for all effect sizes for a given strategy (ie, 34.4%, 58.1%, 27.5%, 43.9% and 18.1% for figure 1A–E, respectively). The upper and lower dotted lines at the edges of the shaded area indicate the 95% confidence band around the predicted effect sizes. The blue vertical lines indicate the last follow-up time point involving at least three studies.