Using routine health information systems for well-designed health evaluations in low- and middle-income countries

Abstract

Routine health information systems (RHISs) are in place in nearly every country and provide routinely collected full-coverage records on all levels of health system service delivery. However, these rich sources of data are regularly overlooked for evaluating causal effects of health programmes due to concerns regarding completeness, timeliness, representativeness and accuracy. Using Mozambique's national RHIS (Módulo Básico) as an illustrative example, we urge renewed attention to the use of RHIS data for health evaluations. Interventions to improve data quality exist and have been tested in low-and middle-income countries (LMICs). Intrinsic features of RHIS data (numerous repeated observations over extended periods of time, full coverage of health facilities, and numerous real-time indicators of service coverage and utilization) provide for very robust quasi-experimental designs, such as controlled interrupted time-series (cITS), which are not possible with intermittent community sample surveys. In addition, cITS analyses are well suited for continuously evolving development contexts in LMICs by: (1) allowing for measurement and controlling for trends and other patterns before, during and after intervention implementation; (2) facilitating the use of numerous simultaneous control groups and non-equivalent dependent variables at multiple nested levels to increase validity and strength of causal inference; and (3) allowing the integration of continuous 'effective dose received' implementation measures. With expanded use of RHIS data for the evaluation of health programmes, investments in data systems, health worker interest in and utilization of RHIS data, as well as data quality will further increase over time. Because RHIS data are ministry-owned and operated, relying upon these data will contribute to sustainable national capacity over time.

Keywords: Epidemiology; evaluation; health information systems; health policy; health systems research; impact; implementation; methods; research methods; survey methods.

Figure 1

Diagrams of potential health programme or policy evaluation research designs using routine community survey data (A) vs RHIS data (B/C). Data observations over time are represented by Os; X represents a programme or policy intervention of interest; A represents an outcome of interest that the programme or policy intervention is expected to affect; B represents a non-equivalent dependent variable not expected to change as a result of the intervention but expected to respond to other potential confounding factors in a similar way as the variable of interest. Adapted from Shadish et al. 2002.

Figure 2

Time-series of number of institutional births from January 2002 to June 2013 for two control clinics and one clinic that underwent intervention to increase institutional births in July 2009. Data are from the national RHIS in Mozambique, the Módulo Básico.

Figure 3

Simple controlled pre-post or difference-in-differences approach with linearity assumption (left) vs cITS design (right). Adapted from Wagenaar and Burris (2013).