Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Feb;8(1):100779.
doi: 10.1016/j.esmoop.2022.100779. Epub 2023 Feb 1.

Design, power, and alpha levels in randomized phase II oncology trials

A Haslam  1 , T Olivier  2 , V Prasad  3
Affiliations
Review

Design, power, and alpha levels in randomized phase II oncology trials

A Haslam et al. ESMO Open. 2023 Feb.

Abstract

Background: The statistical plan of a phase II trial should balance minimizing the premature termination of potentially beneficial therapies (i.e. false negatives) and the further, costly testing of ineffective drugs (i.e. false positives). We sought to examine the methodology, reporting, and bias in the interpretation of outcomes of phase II oncology trials in recent years.

Materials and methods: In a retrospective cross-sectional analysis, we reviewed all full-length articles published on PubMed from 1 January 2021 to 20 June 2022. We searched for data regarding the sample size calculation (number, α value, power, and expected effect size), the primary and secondary outcomes and results, and the authors' conclusion of the study.

Results: About 5.4% of studies (n = 10) used a statistical power that was inferior to 80%, and 16.7% (n = 34) did not indicate the level of power for the sample size calculation. Approximately 16.7% (n = 31) of studies used a one-sided α level of ≤0.025; 17.7% (n = 33) of studies used a predefined threshold (no comparator effect size or difference between groups) to determine the sample size for efficacy. The percentage of studies with a positive authors' conclusion but not meeting the primary endpoint, or the endpoint was equivocal, was 27.4% (n = 51).

Conclusion: Many randomized phase II studies in oncology failed to report essential data for determining sample size calculations, many did not actually use a comparator to determine efficacy even though the studies were randomized, and many had positive conclusions even though the results were indeterminate or the primary endpoint was not met.

Keywords: clinical trials; reporting; sample size calculations.

PubMed Disclaimer

Conflict of interest statement

Disclosure VP has received research funding from Arnold Ventures; royalties from Johns Hopkins Press, Medscape, and MedPage; honoraria for grand rounds/lectures from universities, medical centers, nonprofits, and professional societies; provides consulting for UnitedHealthcare and OptumRX; his Plenary Session podcast (Patreon backers), YouTube, and Substack generate royalties. All other authors have declared no conflicts of interest.

Figures

Figure 1
Figure 1
Flowchart of search strategy for randomized phase II oncology trials.
Figure 2
Figure 2
Association between a study meeting its primary endpoint and the tone of the authors’ conclusion in randomized phase II oncology trials (N = 186). Green indicates where the frequency is greater than expected, and red indicates where the frequency is less than expected.
Figure 3
Figure 3
Frequency of statistical and reporting biases in randomized phase II oncology trials (N = 186).
See this image and copyright information in PMC

References

    1. Rubinstein L.V., Korn E.L., Freidlin B., Hunsberger S., Ivy S.P., Smith M.A. Design issues of randomized phase II trials and a proposal for phase II screening trials. J Clin Oncol. 2005;23(28):7199–7206. - PubMed
    1. Grellety T., Petit-Monéger A., Diallo A., Mathoulin-Pelissier S., Italiano A. Quality of reporting of phase II trials: a focus on highly ranked oncology journals. Ann Oncol. 2014;25(2):536–541. - PubMed
    1. Wayant C., Margalski D., Vaughn K., Vassar M. Evaluation of spin in oncology clinical trials. Crit Rev Oncol Hematol. 2019;144 - PubMed
    1. Chiu K., Grundy Q., Bero L. “Spin” in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9) - PMC - PubMed
    1. Saint S., Christakis D.A., Saha S., et al. Journal reading habits of internists. J Gen Intern Med. 2000;15(12):881–884. - PMC - PubMed

Publication types