Bayesian estimation of HIV acquisition dates for prevention trials

Abstract

Accurate timing estimates of when participants acquire HIV in HIV prevention trials are necessary for determining antibody levels at acquisition. The Antibody-Mediated Prevention (AMP) Studies showed that a passively administered broadly neutralizing antibody can prevent the acquisition of HIV from a neutralization-sensitive virus. We developed a pipeline for estimating the date of detectable HIV acquisition (DDA) in AMP Study participants using diagnostic and viral sequence data. Using a Bayesian strategy that combines three streams of data (REN [rev/vpu/env/Δnef] sequence, GP [gag/Δpol] sequence, and diagnostic) where their 95% credible intervals overlap based on pre-specified criteria and decision rules. We evaluated the performance of our AMP pipeline using PacBio viral sequence data from 41 participants across two prospective acute HIV acquisition cohort studies, FRESH and RV217, with twice-weekly sampling. These cohort studies enrolled young women in South Africa and men and women in Kenya and Thailand, respectively, with a high likelihood of HIV acquisition. In evaluating performance, "true DDA" was the center of bounds between last-negative and first-positive RNA diagnostic tests (median time 4 days, range 2-7 days); bias was the mean difference between estimated and true DDA. Using diagnostic data alone yielded timing estimates with a bias of 2.4 days and root mean square error (RMSE) of 7.9 days. These results were improved using sequence + diagnostic data (bias 1.5 days, RMSE 6.9 days), as well as by restricting sequence-based estimation to samples from ≤5 weeks post-DDA (bias 0.2 days, RMSE 7.8 days).IMPORTANCEIn HIV prevention trials, accurate timing estimates of when individual participants acquire HIV can be used to estimate antibody levels at the time of acquisition, which is useful for projecting antibody levels needed for prevention. The results we report here suggest that if sequence-based estimation of acquisition timing is used in future clinical trials of combination broadly neutralizing antibody (bnAb) regimens or multispecific bnAbs for HIV prevention, a sampling frequency of at least monthly is needed. Moreover, in the samples analyzed here, we observed less bias in sequence-based timing estimation for samples taken <5 weeks post-DDA. This observation is consistent with the timing of immune-driven selective pressures that may negatively impact the power to detect acquisition sieve effects.

Keywords: Antibody-Mediated Prevention (AMP) Studies; Bayesian posterior distribution; FRESH; HIV; RV217; acute acquisition cohort; date of detectable acquisition.

Fig 1

(A) Schematic overview of the types of data input, decision-making steps, data combining, and resultant output of the HIV acquisition timing methods used here. (B) Decision tree diagram employed for the AMP Studies and a guide to subsequent results shown from FRESH and RV217 data analysis to match AMP.

Fig 2

Performance of the six different HIV acquisition timing estimation approaches for the 41 selected FRESH and RV217 participants and the effect of sampling time and HIV serostatus on performance. (A, B) Heatmaps showing (A) bias in days (average error) and (B) RMSE (root mean square error) for six different approaches of estimating date of detectable acquisition (DDA) for FRESH and RV217 participants. In (A), a negative value means an underestimation of the DDA, that is, the given estimator estimated the number of days that the sample was taken from the DDA to be fewer than those yielded by the ground truth COB estimate, and a positive value means an overestimation of the DDA, that is, the given estimator estimated the number of days that the sample was taken from the DDA to be greater than those yielded by the ground truth COB estimate. Rows denote the method used as follows (bottom to top): GP: sequence-based estimation using the GP region; REN: sequence-based estimation using the REN region; SEQ (All): combined sequence-based estimate using both regions, with no restriction on the length of time after which the sample was taken post-COB; SEQ (first 5 weeks): combined sequence-based estimate (GP and REN regions) using only data from the first 5 weeks after COB; DX: diagnostic-based estimation alone; Combined: combined diagnostic and sequence-based estimation. Stacked panels denote (A) the bias or (B) RMSE of various subsets of the data as follows (left to right): (i) overall; (ii) sampled 1–14 days after COB; (iii) sampled 15–28 days after COB; (iv) sampled more than 28 days after COB; (v) HIV antibody-negative (Ab−) sampling periods; and (vi) HIV antibody-positive (Ab+) sampling periods. Note that the 1–14, 15–28, and 29+ bins pool across Ab− and Ab +participants. (C, D, E) Scatterplots of (C) SEQ estimates, (D) DX estimates, and (E) combined estimates of DDA. Approach used in AMP vs actual known COB between last-negative and first-positive HIV test result (most accurate estimate of DDA available from actual acute acquisition study data). In (C) and (D), darker points depict estimates using data sampled more than 5 weeks from COB. In (E), the combined estimate is shown as orange circles when only the DX estimate is used, and as purple crosses when the DX and SEQ estimates are combined. In (C–E), the dotted line is a line of equality and the solid gray line is a regression line.

Fig 3

Comparison of GP vs. REN sequence-based timing estimates for the 41 selected FRESH and RV217 participants. (A) Boxplots, overlaid with individual data of estimation error in days obtained using REN (top, red) vs. GP (bottom, blue) sequence data. Gray lines between dots connect the two sequence-based results from the same sample. For all estimators of DDA, we defined the estimation error as the difference in days between the estimator and the ground truth estimate using the COB between the actual LN and FP dates in RV217 and FRESH cohort data. Positive values indicate overestimation of the number of days between DDA and FP (i.e., the estimate of DDA is too early compared to COB), and negative values indicate underestimation (i.e., the estimate of DDA is too late). (B) Concordance between GP and REN acquisition timing estimates of days since DDA. (C, D) Scatterplots of (C) REN-based estimates of DDA and (D) GP-based estimates of DDA versus the COB DDA estimate. In panels B–D, the dotted line is a line of equality, and the solid gray line is a regression line.

Fig 4

Effects of sampling time and HIV serostatus on SEQ-based DDA estimates for the 41 selected FRESH and RV217 participants. Boxplots, overlaid with estimation error in days obtained using (A) For both serostatuses pooled (Ab− and Ab+), results are stratified by sampling interval post-COB (1–14 days, 15–28 days, and 29+ days); (B) for all sampling times post-COB pooled, results are stratified by HIV serostatus (Ab− vs. Ab+). The dotted vertical line represents perfect estimation of DDA in terms of the ground truth (COB). In interior box plots, left and right vertical edges show the 25th and 75th percentiles of the SEQ estimation error, the middle line indicates the median, and vertical bars extend to the outermost data points within 1.5 times the IQR from these quartiles. For all estimators of DDA, we defined the estimation error as the difference in days between the estimator and the ground truth estimate using the COB between the actual LN and FP dates in RV217 and FRESH cohort data. Positive values indicate overestimation of the number of days between DDA and FP (i.e., the estimate of DDA is too early compared to COB), and negative values indicate underestimation (i.e., the estimate of DDA is too late).

Fig 5

Effects of sampling time and HIV serostatus on DX-based DDA estimates for the 41 selected FRESH and RV217 participants. Boxplots, overlaid with estimation error in days obtained using (A) For both serostatuses pooled (Ab− and Ab+), results are stratified by sampling interval post-COB (1–14 days, 15–28 days, and 29+ days); (B) for all sampling times post-COB pooled, results are stratified by HIV serostatus (Ab− vs. B+). The dotted vertical line represents perfect estimation of DDA in terms of the ground truth (COB). In interior box plots, left and right vertical edges show the 25th and 75th percentiles of the DX estimation error, the middle line indicates the median, and vertical bars extend to the outermost data points within 1.5 times the IQR from these quartiles. For all estimators of DDA, we defined the estimation error as the difference in days between the estimator and the ground truth estimate using the COB between the actual last negative (LN) and first positive (FP) dates in RV217 and FRESH cohort data. Positive values indicate overestimation of the number of days between DDA and FP (i.e., the estimate of DDA is too early compared to COB), and negative values indicate underestimation (i.e., the estimate of DDA is too late).

Fig 6

Effects of sampling time and HIV serostatus on combined-based DDA estimates for the 41 selected FRESH and RV217 participants. Boxplots, overlaid with estimation error in days obtained using (A) For both serostatuses pooled (Ab−- and Ab+), results are stratified by sampling interval post-COB (1–14 days, 15–28 days, 29+ days); (B) for all sampling times post-COB pooled, results are stratified by HIV serostatus (Ab− vs. Ab+). The dotted vertical line represents perfect estimation of DDA in terms of the ground truth (COB). In interior box plots, left and right vertical edges show the 25th and 75th percentiles of the Combined estimation error, the middle line indicates the median, and vertical bars extend to the outermost data points within 1.5 times the IQR from these quartiles. For all estimators of DDA, we defined the estimation error as the difference in days between the estimator and the ground truth estimate using the COB between the actual LN and FP dates in RV217 and FRESH cohort data. Positive values indicate overestimation of the number of days between DDA and FP (i.e., the estimate of DDA is too early compared to COB), and negative values indicate underestimation (i.e., the estimate of DDA is too late).

Fig 7

Individual-level plots showing results of 5 of the DDA estimators for eight participants whose COB was less than 5 weeks from the post-acquisition sampling date. Plots show point estimates and 95% CIs for the GP (blue), REN (red), SEQ (green), DX (yellow), and Combined (purple) DDA estimators. For each sampled observation, the COB is shown as a vertical dotted line, and the number of days that the COB is distant from the sampling date is provided at the top of each plot. HIV serostatus is designated as follows: square, Ab+; diamond, Ab−.

Fig 8

Individual-level plots showing results of five of the DDA estimators for 6 of the studied RV217 participants whose COB was 5 weeks or further from the post-acquisition sampling date. All participants shown had antibody-positive HIV serostatus. Plots show point estimates and 95% CIs for the GP (blue), REN (red), SEQ (green), DX (yellow), and Combined (purple) DDA estimators. For each sampled observation, the COB is shown as a vertical dotted line, and the number of days that the COB is distant from the sampling date is provided at the top of each plot.

Fig 9

Machine learning cross-validation (CV) evaluation of the ability of the Combined and DX estimators to estimate the DDA. Plots show original vs. cross-validation calibrated estimates of the DDA obtained using (A, B) the Combined estimator and (C, D) the DX estimator. In panels A and C, the estimated number of days that the sequenced sample was collected after the COB is plotted against the estimated number of days that the sequenced sample was collected since the DDA. Gray open circles represent original estimates for each estimator; black cross symbols represent CV-calibrated estimates for each estimator. In panels B (Combined estimator) and D (DX estimator), the estimated number of days that the sequenced sample was collected after the DDA (original estimate) is plotted against the estimated number of days that the sequenced sample was collected after the DDA (calibrated estimate). CV was done by linear regression with the intercept fixed at the origin (see Materials and Methods). COB, center of bounds between last-negative and first-positive RNA diagnostic tests (ground truth); DDA, date of detectable acquisition; RMSE, root mean square error.