An evaluation of confidence intervals for a cumulative proportion to enable decisions at interim reviews of single-arm trials

Abstract

Background: Clinical trials often include interim analyses of the proportion of participants experiencing an event by a fixed time-point. A pre-specified proportion excluded from a corresponding confidence interval (CI) may lead an independent monitoring committee to recommend stopping the trial. Frequently this cumulative proportion is estimated by the Kaplan-Meier estimator with a Wald approximate CI, which may have coverage issues with small samples.

Methods: We reviewed four alternative CI methods for cumulative proportions (Beta Product Confidence Procedure (BPCP), BPCP Mid P, Rothman-Wilson, Thomas-Grunkemeier) and two CI methods for simple proportions (Clopper-Pearson, Wilson). We conducted a simulation study comparing CI methods across true event proportions for 12 scenarios differentiated by sample sizes and censoring patterns. We re-analyzed interim data from A5340, a HIV cure trial considering the proportion of participants experiencing virologic failure.

Results: Our simulation study highlights the lower and upper tail error probabilities for each CI method. Across scenarios, we found differences in the performance of lower versus upper bounds. No single method is always preferred. The upper bound of a Wald approximate CI performed reasonably with some error inflation, whereas the lower bound of the BPCP Mid P method performed well. For a trial design similar to A5340, we recommend BPCP Mid P.

Conclusions: The design of future single-arm interim analyses of event proportions should consider the most appropriate CI method based on the relevant bound, anticipated sample size and event proportion. Our paper summarizes available methods, demonstrates performance in a simulation study, and includes code for implementation.

Trial registration: ClinicalTrials.gov NCT02463227.

Keywords: Clinical trials; Clinical trials data monitoring committees; Confidence intervals; Cumulative proportion; Interim review; Single-arm trial.

Figure 1:

Swimmer Plot (A) and Kaplan-Meier Curve (B) of Virologic Failure at the Interim Review for the Motivating Example

Figure 2:. Confidence Interval (CI) for the Cumulative Proportion of Virologic Failure by Each Method at the Interim Review for the Motivating Example

The figure shows the cumulative proportion and simple proportion point estimates together with the corresponding respective 90% CIs. The 0.65 lower bound threshold for considering to stop the trial is indicated by a dotted line.

Figure 3:

Simulations Results for a Uniform Censoring Mechanism Across the Range of Cumulative Probabilities Using a Two-sided Alpha of 0.10

Figure 4:

Simulations Results for a Linear Upward Censoring Mechanism Across the Range of Cumulative Probabilities Using a Two-sided Alpha of 0.10

Figure 5:

Simulations Results for a Linear Downward Censoring Mechanism Across the Range of Cumulative Probabilities Using a Two-sided Alpha of 0.10