Survey of women's report for 33 maternal and newborn indicators: EN-BIRTH multi-country validation study

Abstract

Background: Population-based household surveys, notably the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), remain the main source of maternal and newborn health data for many low- and middle-income countries. As part of the Every Newborn Birth Indicators Research Tracking in Hospitals (EN-BIRTH) study, this paper focuses on testing validity of measurement of maternal and newborn indicators around the time of birth (intrapartum and postnatal) in survey-report.

Methods: EN-BIRTH was an observational study testing the validity of measurement for selected maternal and newborn indicators in five secondary/tertiary hospitals in Bangladesh, Nepal and Tanzania, conducted from July 2017 to July 2018. We compared women's report at exit survey with the gold standard of direct observation or verification from clinical records for women with vaginal births. Population-level validity was assessed by validity ratios (survey-reported coverage: observer-assessed coverage). Individual-level accuracy was assessed by sensitivity, specificity and percent agreement. We tested indicators already in DHS/MICS as well as indicators with potential to be included in population-based surveys, notably the first validation for small and sick newborn care indicators.

Results: 33 maternal and newborn indicators were evaluated. Amongst nine indicators already present in DHS/MICS, validity ratios for baby dried or wiped, birthweight measured, low birthweight, and sex of baby (female) were between 0.90-1.10. Instrumental birth, skin-to-skin contact, and early initiation of breastfeeding were highly overestimated by survey-report (2.04-4.83) while umbilical cord care indicators were massively underestimated (0.14-0.22). Amongst 24 indicators not currently in DHS/MICS, two newborn contact indicators (kangaroo mother care 1.00, admission to neonatal unit 1.01) had high survey-reported coverage amongst admitted newborns and high sensitivity. The remaining indicators did not perform well and some had very high "don't know" responses.

Conclusions: Our study revealed low validity for collecting many maternal and newborn indicators through an exit survey instrument, even with short recall periods among women with vaginal births. Household surveys are already at risk of overload, and some specific clinical care indicators do not perform well and may be under-powered. Given that approximately 80% of births worldwide occur in facilities, routine registers should also be explored to track coverage of key maternal and newborn health interventions, particularly for clinical care.

Keywords: Accuracy; Birth; Coverage; Indicators; Maternal; Newborn; Survey; Validity.

Fig. 1

Survey validation design, EN-BIRTH study. Exact wording for survey questions detailed in Additional file 1

Fig. 2

List of indicators tested for validity, EN-BIRTH study

Fig. 3

Flow Diagram: a Labour and Delivery b Kangaroo Mother Care c Neonatal Infection, EN-BIRTH study

Fig. 4

Coverage for selected indicators, EN-BIRTH study. ¹These indicators are not interventions and prevalence is reported for these indicators. ²Not asked in Tanzania. ³"Don’t Know" is excluded from numerator and denominator. Validity ratio calculated as survey-coverage/observed coverage. Observed data: labour and delivery ward n = 16,030 women, 16,298 newborns; kangaroo mother care n = 840; neonatal infection n = 1015. Survey data: labour and delivery ward n = 14,543 women; kangaroo mother care n = 652; neonatal infection n = 910

Fig. 4

Coverage for selected indicators, EN-BIRTH study. ¹These indicators are not interventions and prevalence is reported for these indicators. ²Not asked in Tanzania. ³"Don’t Know" is excluded from numerator and denominator. Validity ratio calculated as survey-coverage/observed coverage. Observed data: labour and delivery ward n = 16,030 women, 16,298 newborns; kangaroo mother care n = 840; neonatal infection n = 1015. Survey data: labour and delivery ward n = 14,543 women; kangaroo mother care n = 652; neonatal infection n = 910

Fig. 5

Individual-level validation in exit survey for selected indicators, EN-BIRTH study. ¹Validation not done because “Don’t Know” response > 20%. ²Validation not done because ten or fewer observations per column of the two-way table. ³Not asked in Tanzania. Observed data: labour and delivery ward n = 16,030 women, 16,298 newborns; kangaroo mother care n = 840; neonatal infection n = 1015. Survey data: labour and delivery ward n = 14,543 women; kangaroo mother care n = 652; neonatal infection n = 910