Evaluating clinical trial designs for investigational treatments of Ebola virus disease

Abstract

Background: Experimental treatments for Ebola virus disease (EVD) might reduce EVD mortality. There is uncertainty about the ability of different clinical trial designs to identify effective treatments, and about the feasibility of implementing individually randomised controlled trials during an Ebola epidemic.

Methods and findings: A treatment evaluation programme for use in EVD was devised using a multi-stage approach (MSA) with two or three stages, including both non-randomised and randomised elements. The probabilities of rightly or wrongly recommending the experimental treatment, the required sample size, and the consequences for epidemic outcomes over 100 d under two epidemic scenarios were compared for the MSA, a sequential randomised controlled trial (SRCT) with up to 20 interim analyses, and, as a reference case, a conventional randomised controlled trial (RCT) without interim analyses. Assuming 50% 14-d survival in the population treated with the current standard of supportive care, all designs had similar probabilities of identifying effective treatments correctly, while the MSA was less likely to recommend treatments that were ineffective. The MSA led to a smaller number of cases receiving ineffective treatments and faster roll-out of highly effective treatments. For less effective treatments, the MSA had a high probability of including an RCT component, leading to a somewhat longer time to roll-out or rejection. Assuming 100 new EVD cases per day, the MSA led to between 6% and 15% greater reductions in epidemic mortality over the first 100 d for highly effective treatments compared to the SRCT. Both the MSA and SRCT led to substantially fewer deaths than a conventional RCT if the tested interventions were either highly effective or harmful. In the proposed MSA, the major threat to the validity of the results of the non-randomised components is that referral patterns, standard of care, or the virus itself may change during the study period in ways that affect mortality. Adverse events are also harder to quantify without a concurrent control group.

Conclusions: The MSA discards ineffective treatments quickly, while reliably providing evidence concerning effective treatments. The MSA is appropriate for the clinical evaluation of EVD treatments.

Fig 1. Three possible study designs.

Design 1 uses an RCT with a predetermined number of participants, so time to roll-out or rejection of the treatment is determined only by the rate of enrolment of patients into the study. In Design 2, an interim analysis is performed for each additional group of 25 patient outcomes (group SRCT). For Design 3, if the single-arm phase II study indicates a large beneficial effect (conclusion a), the treatment is rolled out, and a phase III single-arm confirmation study is performed. Roll-out continues if this confirmation study gives a positive result; otherwise, an SRCT is employed. In the event of evidence of a moderate effect from the phase II study (conclusion b), the SRCT is employed. The treatment is rejected in the event of no evidence of benefit in the phase II study or negative results from an SRCT. In Designs 2 and 3, the number of participants recruited depends on the outcomes for patients already enrolled, and the study duration is uncertain (as indicated by the colour gradient).

Fig 2. Stopping rules for the component trials of the multi-stage approach.

(A) The single-arm phase II design, (B) the phase III single-arm confirmatory design, and (C) the phase III SRCT.

Fig 3. Comparison of the three designs for evaluating treatments assuming 50% 14-d survival probability in the standard care group.

The designs compared are a conventional RCT without interim analysis (red), a group SRCT with interim analyses for every 25 patients with 14-d outcomes (green), and an MSA (blue). (A) Probability of concluding an experimental treatment is effective (defined as a treatment where the probability of surviving to day 14 is greater than 0.5). (B) Mean time to rolling out or discarding the treatment (solid and dotted lines) and associated 5th and 95th percentiles (shaded regions). In the MSA, a phase III single-arm confirmation study may be performed following roll-out. Time to reach a conclusion from this study is shown by the dashed line. (C) Probability that an RCT is performed in the MSA. (D) Mean number of deaths amongst patients enrolled for the three study designs. (E) Mean percentage reduction in mortality amongst EVD cases in the wider population over 100 d following the start of evaluation compared with no treatment scenarios. Results are shown for two scenarios: one scenario assumes 100 new cases per day (solid lines and filled circles), and the other assumes exponential epidemic growth (dashed lines and open circles). Treatment is assumed to be made available immediately to all new EVD cases after the roll-out time. (F) Mean number of patients starting treatment each day under the scenarios in (E) assuming that the experimental treatment increases 14-d survival probability to 80%.