Effect of imbalanced sampling and missing data on associations between gender norms and risk of adolescent HIV

Abstract

Background: Despite strides towards gender equality, inequalities persist or remain unstudied, due potentially to data gaps. Although mapped, the effects of key data gaps remain unknown. This study provides a framework to measure effects of gender- and age-imbalanced and missing covariate data on gender-health research. The framework is demonstrated using a previously studied pathway for effects of pre-marital sex norms among adults on adolescent HIV risk.

Methods: After identifying gender-age-imbalanced Demographic and Health Survey (DHS) datasets, we resampled responses and restricted covariate data from a relatively complete, balanced dataset derived from the 2007 Zambian DHS to replicate imbalanced gender-age sampling and covariate missingness. Differences in model outcomes due to sampling were measured using tests for interaction. Missing covariate effects were measured by comparing fully-adjusted and reduced model fitness.

Findings: We simulated data from 25 DHS surveys across 20 countries from 2005-2014 on four sex-stratified models for pathways of adult attitude-behaviour discordance regarding pre-marital sex and adolescent risk of HIV. On average, across gender-age-imbalanced surveys, males comprised 29.6% of responses compared to 45.3% in the gender-balanced dataset. Gender-age-imbalanced sampling significantly affected regression coefficients in 40% of model-scenarios (N = 40 of 100) and biased relative-risk estimates away from gender-age-balanced sampling outcomes in 46% (N = 46) of model-scenarios. Model fitness was robust to covariate removal with minor effects on male HIV models. No consistent trends were observed between sampling distribution and risk of biased outcomes.

Interpretation: Gender-health model outcomes may be affected by sampling gender-age-imbalanced data and less-so by missing covariates. Although occasionally attenuated, the effect magnitude of gender-age-imbalanced sampling is variable and may mask true associations, thus misinforming policy dialogue. We recommend future surveys improve balanced gender-age sampling to promote research reliability.

Funding: Bill & Melinda Gates Foundation grant OPP1140262 to Stanford University.

Keywords: Bias; CI, Confidence interval; DHS, Demographic and Health Survey; Data quality; Demographic and health surveys; Gender; Gender data; Gender norms; Global health; HIV; IPV, Intimate partner violence; OR, Odds ratio; PR, Prevalence ratio; RR, Relative risk; SDG, Sustainable Development Goal.

Figure 1

Inclusion and exclusion criteria for gender-balanced and gender-imbalanced datasets. The inclusion and exclusion criteria for identification of a baseline, gender-balanced Demographic and Health Survey (DHS) dataset (panel A) and series of gender-imbalanced DHS datasets (panel B) are provided. As the data informed specific sex-stratified pathways between adult pre-marital sex norms and adolescent risk of HIV, additional criteria unique to the case study were defined [e.g., HIV-gender prevalence ratio (PR) > 2.0].

Figure 2

Unweighted gender distribution of total study-eligible respondents per survey sampled. We calculated the unweighted gender distribution of all respondents per Demographic and Health Survey (gender-balanced and imbalanced) included in the final sample. This included adolescents (15-24 years) with previous sexual experience and HIV testing data, and all adults (24-49 years). The female sample proportions are stacked on top of the male sample proportions. A horizontal line is added to indicate the global sex distribution. Note: Rightmost bar (ZAMBIA) is the gender distribution of the gender-balanced dataset. Key: CBD05: 2005 Cambodia; CDI11: 2011 Cote D'Ivore; CMR11: 2011 Cameroon; DRC07: 2007 Democratic Republic of the Congo; ETH05: 2005 Ethiopia; GAM13: 2013 Gambia; GHA14: 2014 Ghana; GUI05: 2005 Guinea; GUI12: 2012 Guinea; HAI05: 2005 Haiti; HAI12: 2012 Haiti; KEN08: 2008 Kenya; LES09: 2009 Lesotho; LIB13: 2013 Liberia; MAI06: 2006 Mali; MAI12: 2012 Mali; MLW10: 2010 Malawi; NMB13: 2013 Namibia; RWA05: 2005 Rwanda; RWA10: 2010 Rwanda; SEN05: 2005 Senegal; SEN10: 2010 Senegal; SLN08: 2008 Sierra Leone; SLN13: 2013 Sierra Leone; TOG13: 2013 Togo; ZAMBIA: 2007 Zambia.

Figure 3

Unweighted gender-age distribution of study-eligible respondents per survey sampled. We calculated the unweighted gender-age distribution of all respondents per Demographic and Health Survey included in the final sample. Each panel (A-C) consists of the gender-age distribution from eight (panel C has nine) gender-imbalanced datasets, distributed in alphanumeric order, and from the gender-balanced DHS dataset (2007, Zambia; rightmost). Key: CBD05: 2005 Cambodia; CDI11: 2011 Cote D'Ivore; CMR11: 2011 Cameroon; DRC07: 2007 Democratic Republic of the Congo; ETH05: 2005 Ethiopia; GAM13: 2013 Gambia; GHA14: 2014 Ghana; GUI05: 2005 Guinea; GUI12: 2012 Guinea; HAI05: 2005 Haiti; HAI12: 2012 Haiti; KEN08: 2008 Kenya; LES09: 2009 Lesotho; LIB13: 2013 Liberia; MAI06: 2006 Mali; MAI12: 2012 Mali; MLW10: 2010 Malawi; NMB13: 2013 Namibia; RWA05: 2005 Rwanda; RWA10: 2010 Rwanda; SEN05: 2005 Senegal; SEN10: 2010 Senegal; SLN08: 2008 Sierra Leone; SLN13: 2013 Sierra Leone; TOG13: 2013 Togo; ZAMBIA: 2007 Zambia.

Figure 4

Relative risk estimations for the effect of pre-marital sex norms by adults on adolescent female HIV risk when using gender-imbalanced data. We estimated the mean and 95% confidence interval for the association (relative risk) between adolescent female risk of HIV and communal pre-marital sex norms amongst adult females (panel A) and adult men (panel B) using gender-imbalanced sampled datasets. The topmost bar represents the relative risk using the original, gender-balanced sampled dataset. The vertical line at the relative risk of one indicates no association. Key: CBD05: 2005 Cambodia; CDI11: 2011 Cote D'Ivore; CMR11: 2011 Cameroon; DRC07: 2007 Democratic Republic of the Congo; ETH05: 2005 Ethiopia; GAM13: 2013 Gambia; GHA14: 2014 Ghana; GUI05: 2005 Guinea; GUI12: 2012 Guinea; HAI05: 2005 Haiti; HAI12: 2012 Haiti; KEN08: 2008 Kenya; LES09: 2009 Lesotho; LIB13: 2013 Liberia; MAI06: 2006 Mali; MAI12: 2012 Mali; MLW10: 2010 Malawi; NMB13: 2013 Namibia; RWA05: 2005 Rwanda; RWA10: 2010 Rwanda; SEN05: 2005 Senegal; SEN10: 2010 Senegal; SLN08: 2008 Sierra Leone; SLN13: 2013 Sierra Leone; TOG13: 2013 Togo; ZAMBIA: 2007 Zambia.

Figure 5

Relative risk estimations for the effect of pre-marital sex norms of adults on adolescent male HIV risk when using gender-imbalanced data. We estimated the mean and 95% confidence interval for the association (relative risk) between adolescent male risk of HIV and communal pre-marital sex norms amongst adult females (panel A) and adult men (panel B) using gender-imbalanced sampled datasets. The topmost bar represents the relative risk using the original, gender-balanced sampled dataset. The vertical line at the relative risk of one indicates no association. Key: CBD05: 2005 Cambodia; CDI11: 2011 Cote D'Ivore; CMR11: 2011 Cameroon; DRC07: 2007 Democratic Republic of the Congo; ETH05: 2005 Ethiopia; GAM13: 2013 Gambia; GHA14: 2014 Ghana; GUI05: 2005 Guinea; GUI12: 2012 Guinea; HAI05: 2005 Haiti; HAI12: 2012 Haiti; KEN08: 2008 Kenya; LES09: 2009 Lesotho; LIB13: 2013 Liberia; MAI06: 2006 Mali; MAI12: 2012 Mali; MLW10: 2010 Malawi; NMB13: 2013 Namibia; RWA05: 2005 Rwanda; RWA10: 2010 Rwanda; SEN05: 2005 Senegal; SEN10: 2010 Senegal; SLN08: 2008 Sierra Leone; SLN13: 2013 Sierra Leone; TOG13: 2013 Togo; ZAMBIA: 2007 Zambia.