Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Feb 3:3:923.
doi: 10.12688/gatesopenres.12916.2. eCollection 2019.

A systematic review of the agreement of recall, home-based records, facility records, BCG scar, and serology for ascertaining vaccination status in low and middle-income countries

Emily Dansereau  1 , David Brown  2 , Lena Stashko  1 , M Carolina Danovaro-Holliday  1
Affiliations

A systematic review of the agreement of recall, home-based records, facility records, BCG scar, and serology for ascertaining vaccination status in low and middle-income countries

Emily Dansereau et al. Gates Open Res. .

Abstract

Background: Household survey data are frequently used to estimate vaccination coverage - a key indicator for monitoring and guiding immunization programs - in low and middle-income countries. Surveys typically rely on documented evidence from home-based records (HBR) and/or maternal recall to determine a child's vaccination history, and may also include health facility sources, BCG scars, and/or serological data. However, there is no gold standard source for vaccination history and the accuracy of existing sources has been called into question. Methods and Findings: We conducted a systematic review of literature published January 1, 1975 through December 11, 2017 that compared vaccination status at the child-level from at least two sources of vaccination history. 27 articles met inclusion criteria. The percentage point difference in coverage estimates varied substantially when comparing caregiver recall to HBRs (median: +1, range: -43 to +17), to health facility records (median: +5, range: -29 to +34) and to serology (median: -20, range: -32 to +2). Ranges were also wide comparing HBRs to facility-based records (median: +17, range: -61 to +21) and to serology (median: +2, range: -38 to +36). Across 10 studies comparing recall to HBRs, Kappa values exceeded 0.60 in 45% of comparisons; across 7 studies comparing recall to facility-based records, Kappa never reached 0.60. Agreement varied depending on study setting, coverage level, antigen type, number of doses, and child age. Conclusions: Recall and HBR provide relatively concordant vaccination histories in some settings, but both have poor agreement with facility-based records and serology. Long-term, improving clinical decision making and vaccination coverage estimates will depend on strengthening administrative systems and record keeping practices. Short-term, there must be greater recognition of imperfections across available vaccination history sources and explicit clarity regarding survey goals and the level of precision, potential biases, and associated resources needed to achieve these goals.

Keywords: LMIC; agreement; concordance; coverage; immunization; methodology; survey; vaccination; validity.

PubMed Disclaimer

Conflict of interest statement

Competing interests: After completing this analysis and manuscript, ED was hired to work at the Gates Foundation.

Figures

Figure 1.
Figure 1.. Article screening.
Figure 2.
Figure 2.. Comparison of vaccination coverage estimates based on different sources of history.
Figure 3.
Figure 3.. Measures of agreement by source comparison and vaccine.
Figure 4.
Figure 4.. Relationship between coverage level and measures of agreement.
HBR: Home-Based Record, HF: Health Facility, PPV: Positive Predictive Value; NPV: Negative Predictive Value; PP: Percentage Point.
Figure 5.
Figure 5.. Variation in percentage point difference and kappa for different antigens reported in the same study.
Figure 6.
Figure 6.. Variation in percentage point difference and kappa for different doses of the same antigen.
Each point represents a different number of doses for an antigen, and each line connects points for the same antigen, source and study.
Figure 7.
Figure 7.. Variation in percentage point difference and kappa by age group.
Each point represents an age group for a given antigen/dose. Each line connects points for the same antigen/dose, comparison type and study.
Figure 8.
Figure 8.
Relationship between number of doses in national schedule and ( a) percentage point difference in coverage; ( b) kappa.
See this image and copyright information in PMC

References

    1. Duclos P, Okwo-Bele JM, Gacic-Dobo M, et al. : Global immunization: status, progress, challenges and future. BMC Int Health Hum Rights. 2009;9(Suppl 1):S2. 10.1186/1472-698X-9-S1-S2 - DOI - PMC - PubMed
    1. Cutts FT, Claquin P, Danovaro-Holliday MC, et al. : Monitoring vaccination coverage: Defining the role of surveys. Vaccine. 2016;34(35):4103–9. 10.1016/j.vaccine.2016.06.053 - DOI - PMC - PubMed
    1. UNICEF: MICS6 Questionnaires and Modules.2016. Reference Source
    1. WHO: 2015 Working Draft - New WHO Vaccination Coverage Cluster Survey Manual.2015. Reference Source
    1. Demographic and Health Surveys (DHS): DHS Model Questionnaire - Phase 7.2015. Reference Source

Publication types