Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep:209:114230.
doi: 10.1016/j.ejca.2024.114230. Epub 2024 Jul 19.

Challenges of estimating treatment effects after a positive interim analysis

Yu Yang Soon  1 , Ian C Marschner  2 , Manjula Schou  2 , Christopher J Sweeney  3 , Ian D Davis  4 , Martin R Stockler  5 , Andrew J Martin  6
Affiliations
Free article

Challenges of estimating treatment effects after a positive interim analysis

Yu Yang Soon et al. Eur J Cancer. 2024 Sep.
Free article

Abstract

Background: This research investigates why a beneficial treatment effect reported at the first interim analysis (IA) may diminish at a subsequent analysis (SA). We examined three challenges in interpreting treatment effects from randomized clinical trials (RCTs) after the first positive IA: overestimation bias; non-proportional hazards; and heterogeneity in recruitment. We investigate how a penalized estimation method can address overestimation bias, and discuss additional factors to consider when interpreting positive IA results.

Methods: We identified oncology RCTs reporting positive results at the initial IA and a SA for event-free (EFS) and overall survival (OS). We modeled: (1) the hazard ratio at IA (HRIA) versus its timing as measured by the information fraction (IF; i.e., events at IA versus total events sought); and (2), the ratio of HRIA to HRSA (rHR) versus the IF. This was repeated for HRIA adjusted for overestimation bias. Examples of the other two challenges were sought.

Results: Amongst 71 RCTs, HRIA were positively associated with the IF (slope: EFS 0.83, 95 % CI 0.44-1.22; OS 0.25, 95 % CI 0.10-0.41). HRIA tended to exaggerate HRSA, and more so the lower the IF (slope rHR versus IF: EFS 0.10, 95 % CI - 0.22 to 0.42; OS 0.26, 95 % CI 0.07-0.46). Adjusted HRIA did not exaggerate HRSA (slope rHR versus IF: EFS - 0.14, 95 % CI - 0.67 to 0.39; OS 0.02, 95 % CI - 0.26 to 0.30). Examples of two other challenges are shown.

Conclusion: Overestimation bias, non-proportional hazards, and heterogeneity in recruitment and other important treatments should be considered when communicating estimates of treatment effects from positive IAs.

Keywords: Interim analysis; Non-proportional hazards; Overestimation bias; Randomized clinical trials.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Christopher J. Sweeney reports grants from Bayer, Sanofi, Astellas Pharma (Pfizer), Dendreon, and Janssen and personal consulting fees from Bayer, Astellas Pharma, Janssen, CellCentric, Point Biopharma, Pfizer, Novartis, Genentech (Roche), Bristol Myers Squibb, Lilly, Hengrui Europe Biosciences, and AstraZeneca. Martin R. Stockler reports grants from Astellas, Amgen, AstraZeneca, Bayer, Bionomics, Bristol Myers Squibb, Celgene, Medivation, MSD, Pfizer, Roche, Sanofi, and Tilray, all outside the submitted work. Ian D. Davis reports grants from the Australian National Health and Medical Research Council (NHMRC), during the conduct of the study; and institutional payments to support prostate cancer trials from Pfizer, ANZUP Cancer Trials Group, Bayer, Astellas, Janssen, Movember Foundation, and MSD, outside the submitted work. Dr Davis also reports being an unremunerated chair of ANZUP Cancer Trials Group. All other authors declare no competing interests.

LinkOut - more resources