Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2025 Dec 8;27(1):30.
doi: 10.1186/s13063-025-09324-5.

Multiplicity adjustment approaches in publicly funded multi-arm trials: a comprehensive review of the National Institute for Health and Care Research (NIHR) Journals Library

Ellen C Lee  1 , Richard M Jacques  2 , Rebecca M Simpson  2 , Stephen J Walters  2
Affiliations
Review

Multiplicity adjustment approaches in publicly funded multi-arm trials: a comprehensive review of the National Institute for Health and Care Research (NIHR) Journals Library

Ellen C Lee et al. Trials. .

Abstract

Background: Parallel-group multi-arm trials are randomised controlled trials (RCTs) where participants are allocated to three or more concurrent treatment groups. Multiplicity occurs when several statistical tests are conducted within the same study. Statistical adjustments to the design and analysis of multi-arm trials can be used to control the study-wise type I error rate. There is no clear guidance or consensus on the necessity of multiplicity adjustment in multi-arm trials, nor on which methods are most appropriate. This comprehensive review aimed to investigate the design, analysis and reporting of publicly funded parallel-group multi-arm trials and to report the approach to multiplicity in these trials with respect to sample size and statistical analysis.

Methods: We searched the United Kingdom's National Institute for Health and Care Research (NIHR) online Journals Library, from 1 January 1997 to 31 December 2024 for reports of multi-arm RCTs. Information on the trial characteristics, the sample size estimation and analysis of the primary outcome was extracted. Two researchers conducted the search and selected reports for inclusion. Data from each report was independently extracted by two reviewers, and any disagreement was resolved by discussion.

Results: A total of 2452 reports, published online in the NIHR Journals Library, were screened for eligibility; 97 reports of multi-arm parallel-group trials met the inclusion criteria. Of these, 90 included the results of a multi-arm efficacy analysis. In the review, 35% (34/97) of the trials did adjust for multiplicity in the sample size calculation; in 84% (76/90), the potential between-arm comparisons were described in the methods, and 37% (33/90) made a multiplicity adjustment in the analysis. A further 86% (77/86) reported 95% confidence intervals. For the minority of multi-arm trials that did adjust for multiplicity, the most common adjustment method was Bonferroni.

Conclusions: The majority of the publicly funded multi-arm trials did not adjust for multiplicity in the sample size, statistical analysis, or estimation of confidence intervals. Researchers should follow the Consolidated Standards of Reporting Trials guidelines for multi-arm trials and clearly state in protocols and trial reports whether a multiplicity adjustment was made or provide a reason if no adjustment was made.

Keywords: Multi-arm trial; Multiple testing; Multiplicity; Randomised controlled trial; Review; Type 1 error.

PubMed Disclaimer

Conflict of interest statement

Declarations. Ethics approval and consent to participate: Not applicable. Consent for publication: Not applicable. Competing interests: The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Flow diagram showing the search and selection process of RCTs from the six online journals of the NIHR Journals Library surveyed from 1 st January 1997 to 31st December 2024
Fig. 2
Fig. 2
Consistency between multiplicity approach used in sample size calculation and statistical analysis (N=90)
Fig. 3
Fig. 3
Multiplicity adjustment approach over time for the multi-arm trials included in this review (N = 90)
See this image and copyright information in PMC

References

    1. Pike K, Reeves BC, Rogers CA. Approaches to multiplicity in publicly funded pragmatic randomised controlled trials: a survey of clinical trials units and a rapid review of published trials. BMC Med Res Methodol. 2022;22(1):1–11. - DOI - PMC - PubMed
    1. Parmar MKB, Carpenter J, Sydes MR. More multiarm randomised trials of superiority are needed. Lancet. 2014;384(9940):283–4. - DOI - PubMed
    1. Juszczak E, Altman DG, Hopewell S, Schulz K. Reporting of multi-arm parallel-group randomized trials: extension of the CONSORT 2010 statement. JAMA. 2019;321(16):1610. - DOI - PubMed
    1. Odutayo A, Gryaznov D, Copsey B, Monk P, Speich B, Roberts C, et al. Design, analysis and reporting of multi-arm trials and strategies to address multiple testing. Int J Epidemiol. 2020;49(3):968–78. - DOI - PubMed
    1. Wason JMS, Stecher L, Mander AP. Correcting for multiple-testing in multi-arm trials: Is it necessary and is it done? Trials. 2014;15(1):364. - DOI - PMC - PubMed

MeSH terms