Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jan;20(1):109-116.
doi: 10.1002/pst.2059. Epub 2020 Aug 12.

Controlling type I error rates in multi-arm clinical trials: A case for the false discovery rate

James M S Wason  1   2 , David S Robertson  2
Affiliations

Controlling type I error rates in multi-arm clinical trials: A case for the false discovery rate

James M S Wason et al. Pharm Stat. 2021 Jan.

Abstract

Multi-arm trials are an efficient way of simultaneously testing several experimental treatments against a shared control group. As well as reducing the sample size required compared to running each trial separately, they have important administrative and logistical advantages. There has been debate over whether multi-arm trials should correct for the fact that multiple null hypotheses are tested within the same experiment. Previous opinions have ranged from no correction is required, to a stringent correction (controlling the probability of making at least one type I error) being needed, with regulators arguing the latter for confirmatory settings. In this article, we propose that controlling the false-discovery rate (FDR) is a suitable compromise, with an appealing interpretation in multi-arm clinical trials. We investigate the properties of the different correction methods in terms of the positive and negative predictive value (respectively how confident we are that a recommended treatment is effective and that a non-recommended treatment is ineffective). The number of arms and proportion of treatments that are truly effective is varied. Controlling the FDR provides good properties. It retains the high positive predictive value of FWER correction in situations where a low proportion of treatments is effective. It also has a good negative predictive value in situations where a high proportion of treatments is effective. In a multi-arm trial testing distinct treatment arms, we recommend that sponsors and trialists consider use of the FDR.

Keywords: error rate; false discovery rate; family-wise error rate; multi-arm trials.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Positive and negative predictive values estimated from the simulation study with three experimental arms as the proportion of treatments which are truly effective changes.
A, Positive predictive value. B, Negative predictive value
Figure 2
Figure 2. Positive and negative predictive values estimated from the simulation study with five experimental arms as the proportion of treatments which are truly effective changes.
A, Positive predictive value. B, Negative predictive value
See this image and copyright information in PMC

References

    1. Parmar MKB, Carpenter J, Sydes MR. More multiarm randomised trials of superiority are needed. Lancet. 2014;384:283–284. - PubMed
    1. Parmar MKB, Barthel FM-S, Sydes M, et al. Speeding up the evaluation of new agents in cancer. J Natl Cancer Inst. 2008;100:1204–1214. - PMC - PubMed
    1. Jaki T, Wason JMS. Multi-arm multi-stage trials can improve the efficiency of finding effective treatments for stroke: a case study. BMC Cardiovasc Disord. 2018;18(1):215. - PMC - PubMed
    1. Phillips PJ, Gillespie SH, Boeree M, et al. Innovative trials designs are practical solutions for improving the treatment of tuberculosis. J Infect Dis. 2012;205:250–257. - PubMed
    1. Wason J, Magirr D, Law M, Jaki T. Some recommendations for multi-arm multi-stage trials. Stat Methods Med Res. 2016;25(2):716–727. - PMC - PubMed

Publication types