Statistical interpretation of the RV144 HIV vaccine efficacy trial in Thailand: a case study for statistical issues in efficacy trials

Abstract

Recently, the RV144 randomized, double-blind, efficacy trial in Thailand reported that a prime-boost human immunodeficiency virus (HIV) vaccine regimen conferred ∼30% protection against HIV acquisition. However, different analyses seemed to give conflicting results, and a heated debate ensued as scientists and the broader public struggled with their interpretation. The lack of accounting for statistical principles helped flame the debate, and we leverage these principles to provide a more scientific interpretation. We first address interpretation of frequentist results, including interpretation of P values, synthesis of results from multiple analyses (ie, intention-to-treat versus per-protocol/fully immunized), and accounting for external efficacy trials. Second, we address how Bayesian statistics, which provide clearly interpretable statements about probabilities that the vaccine efficacy takes certain values, provide more information for weighing the evidence about efficacy than do frequentist statistics alone. Third, we evaluate RV144 for completeness of end point ascertainment and integrity of blinding, necessary tasks for establishing robustly interpretable results.

Figure 1.

Pr(VE = 0%|RV144 data) and the density P(VE|RV144 data) for nonzero values of vaccine efficacy (VE) when the assumed prior is Pr(VE = 0%) = .5 and Pr(−20% < VE < 60%) = .5 with equal likelihood of all nonzero VE values between −20% and 60%.

Figure 2.

Pr(VE = 0%|RV144 data) as one varies the prior upper limit, VE* (the largest efficacy one might expect before seeing the data), on vaccine efficacy (VE) when the assumed prior has Pr(VE = 0%) adjusted for the three previous HIV vaccine trials and has equal likelihood of all nonzero VE values between –(VE*/3)% and VE*%.