Evaluation of the Fill-it-up-design to use historical control data in randomized clinical trials with two arm parallel group design

Abstract

Purpose: In the context of clinical research, there is an increasing need for new study designs that help to incorporate already available data. With the help of historical controls, the existing information can be utilized to support the new study design, but of course, inclusion also carries the risk of bias in the study results.

Methods: To combine historical and randomized controls we investigate the Fill-it-up-design, which in the first step checks the comparability of the historical and randomized controls performing an equivalence pre-test. If equivalence is confirmed, the historical control data will be included in the new RCT. If equivalence cannot be confirmed, the historical controls will not be considered at all and the randomization of the original study will be extended. We are investigating the performance of this study design in terms of type I error rate and power.

Results: We demonstrate how many patients need to be recruited in each of the two steps in the Fill-it-up-design and show that the family wise error rate of the design is kept at 5 $\%$ . The maximum sample size of the Fill-it-up-design is larger than that of the single-stage design without historical controls and increases as the heterogeneity between the historical controls and the concurrent controls increases.

Conclusion: The two-stage Fill-it-up-design represents a frequentist method for including historical control data for various study designs. As the maximum sample size of the design is larger, a robust prior belief is essential for its use. The design should therefore be seen as a way out in exceptional situations where a hybrid design is considered necessary.

Keywords: Equivalence; External controls; Historical control; Power; Randomized clinical trial; Sample size; Type I error probability.

Fig. 1

Flow chart of the procedure of the Fill-it-up-design

Fig. 2

Family wise error rate testing simultaneously superiority tests ($S_1$) and ($S_2$) for different scenarios of the Fill-it-up-design depending on the choice of the significance level of the equivalence pre-test. A medium effect size $\delta =0.5$ with $n_H=500$ historical controls and an equivalence margin of $\mathrm{\Delta }=0.44$ is examined

Fig. 3

Family wise error rate testing simultaneously superiority tests ($S_1$) and ($S_2$) assuming ${\mu }_E={\mu }_C=0$ depending on the choice of the significance level of the equivalence pre-test. $n_H=500$ historical controls were included considering a small effect $\delta =0.2$ with equivalence margin $\mathrm{\Delta }=0.19$, medium effect $\delta =0.5$ with equivalence margin $\mathrm{\Delta }=0.44$ and large effect $\delta =0.8$ with equivalence margin $\mathrm{\Delta }=0.70$

Fig. 4

Evaluation of type I error and power for MAP approach for full-sample and sub-sample using non-informative, non-robust and robust priors and half normal distribution for heterogeneity