Evaluation of respondent-driven sampling

Abstract

Background: Respondent-driven sampling is a novel variant of link-tracing sampling for estimating the characteristics of hard-to-reach groups, such as HIV prevalence in sex workers. Despite its use by leading health organizations, the performance of this method in realistic situations is still largely unknown. We evaluated respondent-driven sampling by comparing estimates from a respondent-driven sampling survey with total population data.

Methods: Total population data on age, tribe, religion, socioeconomic status, sexual activity, and HIV status were available on a population of 2402 male household heads from an open cohort in rural Uganda. A respondent-driven sampling (RDS) survey was carried out in this population, using current methods of sampling (RDS sample) and statistical inference (RDS estimates). Analyses were carried out for the full RDS sample and then repeated for the first 250 recruits (small sample).

Results: We recruited 927 household heads. Full and small RDS samples were largely representative of the total population, but both samples underrepresented men who were younger, of higher socioeconomic status, and with unknown sexual activity and HIV status. Respondent-driven sampling statistical inference methods failed to reduce these biases. Only 31%-37% (depending on method and sample size) of RDS estimates were closer to the true population proportions than the RDS sample proportions. Only 50%-74% of respondent-driven sampling bootstrap 95% confidence intervals included the population proportion.

Conclusions: Respondent-driven sampling produced a generally representative sample of this well-connected nonhidden population. However, current respondent-driven sampling inference methods failed to reduce bias when it occurred. Whether the data required to remove bias and measure precision can be collected in a respondent-driven sampling survey is unresolved. Respondent-driven sampling should be regarded as a (potentially superior) form of convenience sampling method, and caution is required when interpreting findings based on the sampling method.

Figure 1. Map of study area showing location of target population and seed households and respondent-driven sampling interview sites

Colors are used to represent households in different villages. Each village has been labelled with a letter for confidentiality.

Figure 2. Summary of the dynamics of respondent-driven sampling survey recruitment

(A)The cumulative number of recruits over time (including seeds). (B) The total number of recruits per seed (excluding seeds). (C)The number of recruits by wave and seed (including seeds).(D) The number of days between recruiters’ interview and their recruits’ first interview. (E) The number of recruits per recruiter, overall and by whether the recruiters returned for incentive collection (including seeds). (F) The proportion of recruit's network who had already been recruited at the time of their interview (using network size definition NS-5, including seeds).

Figure 3. Recruitment networks showing HIV infection status, by seed

Seeds are shown at the top of each recruitment network. Symbol area is proportional to network size. HIV serostatus is shown by shading: black indicates HIV positive; white, HIV negative; grey, HIV status unknown. HIV status omitted for seeds for confidentiality.