Structural Sensitivity in HIV Modeling: A Case Study of Vaccination

Abstract

Structural assumptions in infectious disease models, such as the choice of network or compartmental model type or the inclusion of different types of heterogeneity across individuals, might affect model predictions as much as or more than the choice of input parameters. We explore the potential implications of structural assumptions on HIV model predictions and policy conclusions. We illustrate the value of inference robustness assessment through a case study of the effects of a hypothetical HIV vaccine in multiple population subgroups over eight related transmission models, which we sequentially modify to vary over two dimensions: parameter complexity (e.g., the inclusion of age and HCV comorbidity) and contact/simulation complexity (e.g., aggregated compartmental vs. individual/disaggregated compartmental vs. network models). We find that estimates of HIV incidence reductions from network models and individual compartmental models vary, but those differences are overwhelmed by the differences in HIV incidence between such models and the aggregated compartmental models (which aggregate groups of individuals into compartments). Complexities such as age structure appear to buffer the effects of aggregation and increase the threshold of net vaccine effectiveness at which aggregated models begin to overestimate reductions. The differences introduced by parameter complexity in estimated incidence reduction also translate into substantial differences in cost-effectiveness estimates. Parameter complexity does not appear to play a consistent role in differentiating the projections of network models.

Keywords: HIV transmission; HIV vaccine; dynamic compartmental model; inference robustness assessment; network model; structural sensitivity analysis.

Fig. 1

Schematic of model types. PWID = people who inject drugs. MSM = men who have sex with men. HCV = hepatitis C virus. HIV = human immunodeficiency virus. PWUD = people who abuse drugs but do not inject. The eight models in our case study are organized along two dimensions: contact/simulation complexity and parameter complexity. Moving from left to right, along the parameter complexity direction, we begin with three basic risk groups (PWID, MSM, and low risk) and HIV; add age structure and HCV comorbidity; and finally add incarceration structure, the PWUD risk group, and race to the models. Moving from top to bottom, along the contact/simulation complexity dimension, we begin with a close approximation of a dynamic compartmental model; introduce heightened stochasticity; move compartment size from a subpopulation to a (homogeneous) individual; introduce heterogeneity of individuals; and finally include network structure.

Fig. 2

Model calibration: Distribution of projected HIV incidence over five years across subpopulations in the absence of a vaccine intervention. PWID = people who inject drugs. MSM = men who have sex with men. The eight models were calibrated to a number of epidemiologic targets from King County, Washington (Appendix Figures A.1-A.10), including the distribution of HIV incidence across risk groups as shown here.

Fig. 3

Vaccine scenarios in the PWID target population: percent decrease in HIV incidence over five years. PWID = people who inject drugs. The figure presents the results of all vaccine scenarios targeted to the PWID population across the eight models. The figure shows the percent of total HIV infections averted.

Fig. 4

Vaccine scenarios in the PWID/MSM target population: percent decrease in HIV incidence over five years. PWID = people who inject drugs. MSM = men who have sex with men. The figure presents the results of all vaccine scenarios targeted to the PWID/MSM population (that is, individuals who fall into both categories) across the eight models. The figure shows the percent of total HIV infections averted.

Fig. 5

Vaccine scenarios in the MSM target population: percent decrease in HIV incidence over five years. MSM = men who have sex with men. The figure presents the results of all vaccine scenarios targeted to the MSM population across the eight models. The figure shows the percent of total HIV infections averted.