Point estimation for adaptive trial designs II: Practical considerations and guidance

Abstract

In adaptive clinical trials, the conventional end-of-trial point estimate of a treatment effect is prone to bias, that is, a systematic tendency to deviate from its true value. As stated in recent FDA guidance on adaptive designs, it is desirable to report estimates of treatment effects that reduce or remove this bias. However, it may be unclear which of the available estimators are preferable, and their use remains rare in practice. This article is the second in a two-part series that studies the issue of bias in point estimation for adaptive trials. Part I provided a methodological review of approaches to remove or reduce the potential bias in point estimation for adaptive designs. In part II, we discuss how bias can affect standard estimators and assess the negative impact this can have. We review current practice for reporting point estimates and illustrate the computation of different estimators using a real adaptive trial example (including code), which we use as a basis for a simulation study. We show that while on average the values of these estimators can be similar, for a particular trial realization they can give noticeably different values for the estimated treatment effect. Finally, we propose guidelines for researchers around the choice of estimators and the reporting of estimates following an adaptive design. The issue of bias should be considered throughout the whole lifecycle of an adaptive design, with the estimation strategy prespecified in the statistical analysis plan. When available, unbiased or bias-reduced estimates are to be preferred.

Keywords: adaptive design; bias-correction; conditional bias; point estimation; unconditional bias.

FIGURE 1

Bias of the average treatment effect and probability of early stopping for a two‐stage group sequential design using one‐sided O'Brien‐Fleming efficacy stopping boundaries under different sample sizes (small, large, medium), with overall α = 0.05. The interim analysis P‐value threshold for efficacy is 0.0088. (A) The expected over‐estimation in trial realizations that stop early for overwhelming efficacy (ie, conditional bias), (B) The expected under‐estimation in trial realizations that do not stop early for overwhelming efficacy (ie, conditional bias at the final analysis), (C) the overall (unconditional) bias, and (D) the probability of early stopping.

FIGURE 2

Sampling distributions of the point estimates from 10⁵ trial replicates, assuming that θ = 0.14. CMUE, conditional median unbiased estimator; MLE, maximum likelihood estimator; MUE, median unbiased estimator; UMVCUE, uniformly minimum variance conditionally unbiased estimator; UMVUE, uniformly minimum variance unbiased estimator.