Who is missed in a community-based survey: Assessment and implications of biases due to incomplete sampling frame in a community-based serosurvey, Choma and Ndola Districts, Zambia, 2022

Abstract

Community-based serological studies are increasingly relied upon to measure disease burden, identify population immunity gaps, and guide control and elimination strategies; however, there is little understanding of the potential for and impact of sampling biases on outcomes of interest. As part of efforts to quantify measles immunity gaps in Zambia, a community-based serological survey using stratified multi-stage cluster sampling approach was conducted in Ndola and Choma districts in May-June 2022, enrolling 1245 individuals. We carried out a follow-up study among individuals missed from the sampling frame of the serosurvey in July-August 2022, enrolling 672 individuals. We assessed the potential for and impact of biases in the community-based serosurvey by i) estimating differences in characteristics of households and individuals included and excluded (77% vs 23% of households) from the sampling frame of the serosurvey and ii) evaluating the magnitude these differences make on healthcare-seeking behavior, vaccination coverage, and measles seroprevalence. We found that missed households were 20% smaller and 25% less likely to have children. Missed individuals resided in less wealthy households, had different distributions of sex and occupation, and were more likely to seek care at health facilities. Despite these differences, simulating a survey in which missed households were included in the sampling frame resulted in less than a 5% estimated bias in these outcomes. Although community-based studies are upheld as the gold standard study design in assessing immunity gaps and underlying community health characteristics, these findings underscore the fact that sampling biases can impact the results of even well-conducted community-based surveys. Results from these studies should be interpreted in the context of the study methodology and challenges faced during implementation, which include shortcomings in establishing accurate and up-to-date sampling frames. Failure to account for these shortcomings may result in biased estimates and detrimental effects on decision-making.

Fig 1. Status of households enrolled in the original community-based measles serological survey and missed populations study, Ndola and Choma Districts, Zambia, 2022.

A. The distribution of household status from listing in the original serosurvey conducted in Choma and Ndola Districts, by cluster. Households classified as “Available” provided consent to participate in the study and reported that they would be available during the data collection; these households comprised the sampling frame for the original study. Households that refused (“Refused”) were excluded from the original study sampling frame and were ineligible for the missed populations study. Households classified as “Non-contact” were households that were locked at the time of listing (and during revisits), or if there was no adult respondent at home, and nobody was available to provide information about the household (e.g. neighbor). Finally, households that were listed but which reported not being available during data collection (“Contact, not available”) were excluded from the sampling frame in the original study. The households in the latter two categories were eligible for the missed populations study. Clusters are arranged in descending order by percentage of households eligible for the missed populations study (“Non-contact” and “Contact, not available” households). “X”‘s indicate clusters selected for the missed population study. B. Distribution of households that the data collection team attempted to reach by status, cluster, and district in the missed populations study. Households classified as “Completed” were successfully located and provided consent to participate in the study. “Household not found” indicates households identified for inclusion in the missed populations study that could not be located during this study. “Non-contact” refers to households which were physically located, but ones in which the data collection team could not contact its occupants. No household refused participation. Clusters are arranged in order of decreasing percent missed in the missed populations study, comprised of “Household Not Found” and “Non-contact” households.

Fig 2. Vaccination status for children 1–4 years old, by vaccine.

Results are presented for the original community-based measles serological survey and missed populations study, Ndola and Choma Districts, Zambia, 2022. The vaccines considered are bacille Calmette-Guérin, BCG; the first, second, and third dose of diphtheria, pertussis, and tetanus (Penta1, Penta2, Penta3); and first and second doses of measles-containing vaccine (MCV1, MCV2).

Fig 3. Estimates of outcomes of interest using the sampling frame from the original community-based measles serological survey (excluding missed households) in Ndola and Choma Districts, Zambia, 2022, and a mixed sampling frame (including both households enrolled in the missed population study and households enrolled in the original study).

Weighting was done using the estimated population in each age group in each cluster in the missed population study for outcomes of interest including A. Healthcare seeking (actual and theoretical) at facilities of interest (Arthur Davison Children’s Hospital and Choma General Hospital for children 1–4 and 5–14 years old, and Ndola Teaching Hospital and Choma General Hospital for adults 15 years and older), B. MCV2 coverage, children 1–4 years old, and C. Measles seroprevalence, children 1–4 years old.

Fig 4. Amount of bias for estimates of outcome of interest using different missingness thresholds to trigger additional listing, by district.

Outcomes of interest are A. female respondents among adults 15 years and over, and B. households with children under 15 years of age. Bias is defined as the difference in the indicated estimate in the original dataset (community-based measles serological survey carried out in Ndola and Choma Districts, Zambia, 2022), or bootstrapped with a specified percent missingness threshold and the estimate from a dataset with a sampling frame that includes all missed population households.