Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Oct 1;29(10):2135-2139.
doi: 10.1093/annonc/mdy340.

'Optimism bias' in contemporary national clinical trial network phase III trials: are we improving?

Kaveh Zakeri  1 , Sonal Noticewala  2 , Lucas Vitzthum  2 , E Sojourner  2 , Hanjie Shen  3 , Loren Mell  4
Affiliations

'Optimism bias' in contemporary national clinical trial network phase III trials: are we improving?

Kaveh Zakeri et al. Ann Oncol. .

Abstract

Background: Previous studies have found that overestimating treatment effects (i.e. 'optimism bias') leads to underpowered clinical trials. The prevalence of 'optimism bias' in contemporary National Clinical Trials Network (NCTN) cancer clinical trials is unknown.

Methods: We conducted a systematic review of NCTN phase III randomized trials published from January 2007 to January 2017. We compared the hypothesized versus observed treatment effects in each trial, and examined whether trial-related factors were correlated with the study results. We also reviewed the methods of each protocol to assess whether a rationale for the hypothesized effect size was provided.

Results: We identified 161 clinical trials, of which 130 were eligible for analysis. Original protocols could not be located for 8 trials (5.0%). Twenty-eight trials (21.5%) observed a statistically significant difference in the primary end point favoring the experimental treatment. The median ratio of observed-to-expected hazard ratios among trials that observed a statistically significant effect on the primary end point was 1.07 (range: 0.33-1.28) versus 1.32 (range: 0.86-2.02) for trials that did not, compared with 1.34 and 1.86, respectively, for National Cancer Institute (NCI) trials published between 1955 and 2006. An effect size at least as large as the one projected in the protocol trials was observed in 9.8% of trials, compared with 17% of NCI trials published from 1955 to 2006. Most trials (64.6%) provided no rationale to support the magnitude of the proposed treatment effect in the protocol.

Conclusions: Despite a reduction in 'optimism bias' compared with previous eras, most contemporary NCTN phase III trials failed to establish statistically significant benefits of new cancer therapies. Better rationalization of proposed effect sizes in research protocols is needed.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Flowchart for the selection of clinical trials included in the analysis.
Figure 2.
Figure 2.
Kernel density plot of hypothesized versus observed treatment effects on the primary end point, displayed as hazard ratios. Values <1.0 indicate a benefit favoring the experimental therapy.
Figure 3.
Figure 3.
Difference between hypothesized versus observed treatment effects for trials reporting time-to-event outcomes with hazard ratios.
See this image and copyright information in PMC

References

    1. Djulbegovic B, Kumar A, Soares HP. et al. Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Arch Intern Med 2008; 168(6): 632–642. - PMC - PubMed
    1. Djulbegovic B, Kumar A, Glasziou P. et al. Medical research: trial unpredictability yields predictable therapy gains. Nature 2013; 500(7463): 395–396. - PMC - PubMed
    1. Dent L, Raftery J.. Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme. Trials 2011; 12(1): 109.. - PMC - PubMed
    1. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987; 317(3): 141–145. - PubMed
    1. Califf RM, Zarin DA, Kramer JM. et al. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010. JAMA 2012; 307(17): 1838–1847. - PubMed