Using health management information system data: case study and verification of institutional deliveries in Ethiopia

Abstract

Health management information systems (HMIS) are a crucial source of timely health statistics and have the potential to improve reporting in low-income countries. However, concerns about data quality have hampered their widespread adoption in research and policy decisions. This article presents results from a data verification study undertaken to gain insights into the quality of HMIS data in Ethiopia. We also provide recommendations for working with HMIS data for research and policy translation. We linked the HMIS to the 2016 Emergency Obstetric and Newborn Care Assessment, a national census of all health facilities that provided maternal and newborn health services in Ethiopia. We compared the number of visits for deliveries and caesarean sections (C-sections) reported in the HMIS in 2015 (January-December) to those found in source documents (paper-based labour and delivery and operating theatre registers) in 2425 facilities across Ethiopia. We found that two-thirds of facilities had 'good' HMIS reporting for deliveries (defined as reporting within 10% of source documents) and half had 'very good' reporting (within 5% of source documents). Results were similar for reporting on C-section deliveries. We found that good reporting was more common in urban areas (OR: 1.30, 95% CI 1.06 to 1.59), public facilities (OR: 2.95, 95% CI 1.38 to 6.29) and in hospitals compared with health centres (OR: 1.71, 95% CI 1.13 to 2.61). Facilities in the Somali and Afar regions had the lowest odds of good reporting compared with Addis Ababa and were more likely to over-report deliveries in the HMIS. Further work remains to address remaining discrepancies in the Ethiopian HMIS. Nonetheless, our findings corroborate previous data verification exercises in Ethiopia and support greater use and uptake of HMIS data for research and policy decisions (particularly, greater use of HMIS data elements (eg, absolute number of services provided each month) rather than coverage indicators). Increased use of these data, combined with feedback mechanisms, is necessary to maintain data quality.

Keywords: health services research; health systems; health systems evaluation; maternal health.

Figure 1

Ratios of deliveries and C-sections reported in the Ethiopian HMIS to those reported in paper-based facility registers (verification factors) by facilities in 2015. HMIS, health management information systems; C-section, Caesarean section. Ratios of the total deliveries and C-sections reported in the HMIS from January to December 2015 to that extracted from health facility registers for the same months (verification factors (VFs)). A VF of 1 means that the number of deliveries in the HMIS matches that found in registers. A VF > 1 indicates a greater number reported in the HMIS, and a VF < 1 indicates a smaller number reported in the HMIS compared to registers. Histogram on the left is truncated at 0.5 and 2.0 to show the region of the plot with most facilities (147 out of 2,425 facilities (6%) are excluded). The plot on the right excludes one facility with a VF of 4 for C-sections.

Figure 2

Verification factors (total deliveries in HMIS divided by that in registers) by Ethiopian region in 2015. HMIS, health management information systems; SNNPR, Southern Nations, Nationalities, and Peoples’ Region. Boxplots are truncated at 0.5 and 2.0 to show the region of the plot with most facilities (147 of 2,425 facilities (6%) are excluded). Regions are sorted by median VF after truncation. The red line at 1 indicates perfect agreement. The blue dotted lines show the region of “very good” reporting: HMIS is within 5% of registers. The median VF in Benishangul-Gumuz was 1.17 before truncation.