Measuring coverage in MNCH: design, implementation, and interpretation challenges associated with tracking vaccination coverage using household surveys

Abstract

Vaccination coverage is an important public health indicator that is measured using administrative reports and/or surveys. The measurement of vaccination coverage in low- and middle-income countries using surveys is susceptible to numerous challenges. These challenges include selection bias and information bias, which cannot be solved by increasing the sample size, and the precision of the coverage estimate, which is determined by the survey sample size and sampling method. Selection bias can result from an inaccurate sampling frame or inappropriate field procedures and, since populations likely to be missed in a vaccination coverage survey are also likely to be missed by vaccination teams, most often inflates coverage estimates. Importantly, the large multi-purpose household surveys that are often used to measure vaccination coverage have invested substantial effort to reduce selection bias. Information bias occurs when a child's vaccination status is misclassified due to mistakes on his or her vaccination record, in data transcription, in the way survey questions are presented, or in the guardian's recall of vaccination for children without a written record. There has been substantial reliance on the guardian's recall in recent surveys, and, worryingly, information bias may become more likely in the future as immunization schedules become more complex and variable. Finally, some surveys assess immunity directly using serological assays. Sero-surveys are important for assessing public health risk, but currently are unable to validate coverage estimates directly. To improve vaccination coverage estimates based on surveys, we recommend that recording tools and practices should be improved and that surveys should incorporate best practices for design, implementation, and analysis.

Figure 1. Schematic of recording of vaccination data at the time of vaccination and during community surveys.

Recording at the time of vaccination (primary recording) is indicated in black boxes; recording during surveys is indicated in green boxes. Main potential sources of information error and bias are highlighted in blue. DOB, date of birth.

Figure 2. Several instances of improvisation on a vaccination card.

(Photo courtesy of Carolina Danovaro, Pan American Health Organization.)

Figure 3. Operating characteristic curves for four LQAS sampling plans.

In each panel, the curve indicates the probability of finding d* or more vaccinated children in a random sample of size n. Lots with coverage≤lower threshold (LT) will be classified as having inadequate coverage with probability ≥(1 − α). Lots with coverage≥upper threshold (UT) will be classified as having adequate coverage with probability ≥(1 − β). The gray area is the region where LT<coverage<UT; lots with coverage in the gray area may be labeled either adequate or inadequate. The gray area includes the region of coverage, for instance, where there is a 50/50 probability of being classified adequate or inadequate. Neither classification (adequate or inadequate) rules out the strong possibility that the true coverage lies in the gray area. The gray area may be made larger or smaller and may be moved to regions of higher or lower coverage by manipulating LT, UT, α, and β to arrive at different values of n and d*.