Estimating and Testing Vaccine Sieve Effects Using Machine Learning

Abstract

When available, vaccines are an effective means of disease prevention. Unfortunately, efficacious vaccines have not yet been developed for several major infectious diseases, including HIV and malaria. Vaccine sieve analysis studies whether and how the efficacy of a vaccine varies with the genetics of the pathogen of interest, which can guide subsequent vaccine development and deployment. In sieve analyses, the effect of the vaccine on the cumulative incidence corresponding to each of several possible genotypes is often assessed within a competing risks framework. In the context of clinical trials, the estimators employed in these analyses generally do not account for covariates, even though the latter may be predictive of the study endpoint or censoring. Motivated by two recent preventive vaccine efficacy trials for HIV and malaria, we develop new methodology for vaccine sieve analysis. Our approach offers improved validity and efficiency relative to existing approaches by allowing covariate adjustment through ensemble machine learning. We derive results that indicate how to perform statistical inference using our estimators. Our analysis of the HIV and malaria trials shows markedly increased precision -- up to doubled efficiency in both trials -- under more plausible assumptions compared with standard methodology. Our findings provide greater evidence for vaccine sieve effects in both trials.

Keywords: HIV; competing risks; dependent censoring; machine learning; malaria; targeted minimum loss-based estimation; vaccine.

Figure 1:

Results from the TMLE analysis of the RV144 trial. Top: Cumulative incidence in vaccine and control groups of HIV-1 infections matched (left) and mismatched (right) to the vaccine at AA 169. Bottom left: Vaccine efficacy against matched (triangle) and mismatched (circle) infections. Bottom right: Vaccine sieve effect with 95% confidence intervals.

Figure 2:

Relative sieve effect estimate (triangle) and confidence interval width (circle) from RV144 trial comparing TMLE to AJ. Values less than one indicate lower point estimate/variance for TMLE.

Figure 3:

Results from the TMLE analysis of the RTS,S/AS01 trial. Top: Multiple outputation TMLE estimates in vaccine and control groups of the incidence of clinical malaria cases matched (left) and mismatched (right) to the vaccine along the CS protein. Bottom left: Vaccine efficacy against matched (triangle) and mismatched (circle) infections. Bottom right: Vaccine sieve effect with 95% confidence intervals.

Figure 4:

Relative sieve effect estimate (triangle) and confidence interval width (circle) from RTS,S/AS01 trial comparing TMLE to AJ. Values less than one indicate lower point estimate/variance for TMLE.