Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Mar 21;16(1):45.
doi: 10.1186/s12916-018-1023-9.

Comparison of treatment effect sizes from pivotal and postapproval trials of novel therapeutics approved by the FDA based on surrogate markers of disease: a meta-epidemiological study

Joshua D Wallach  1   2 , Oriana Ciani  3   4 , Alison M Pease  5 , Gregg S Gonsalves  6   7 , Harlan M Krumholz  8   9   10 , Rod S Taylor  3 , Joseph S Ross  6   8   10   11
Affiliations

Comparison of treatment effect sizes from pivotal and postapproval trials of novel therapeutics approved by the FDA based on surrogate markers of disease: a meta-epidemiological study

Joshua D Wallach et al. BMC Med. .

Abstract

Background: The U.S. Food and Drug Administration (FDA) often approves new drugs based on trials that use surrogate markers for endpoints, which involve certain trade-offs and may risk making erroneous inferences about the medical product's actual clinical effect. This study aims to compare the treatment effects among pivotal trials supporting FDA approval of novel therapeutics based on surrogate markers of disease with those observed among postapproval trials for the same indication.

Methods: We searched Drugs@FDA and PubMed to identify published randomized superiority design pivotal trials for all novel drugs initially approved by the FDA between 2005 and 2012 based on surrogate markers as primary endpoints and published postapproval trials using the same surrogate markers or patient-relevant outcomes as endpoints. Summary ratio of odds ratios (RORs) and difference between standardized mean differences (dSMDs) were used to quantify the average difference in treatment effects between pivotal and matched postapproval trials.

Results: Between 2005 and 2012, the FDA approved 88 novel drugs for 90 indications based on one or multiple pivotal trials using surrogate markers of disease. Of these, 27 novel drugs for 27 indications were approved based on pivotal trials using surrogate markers as primary endpoints that could be matched to at least one postapproval trial, for a total of 43 matches. For nine (75.0%) of the 12 matches using the same non-continuous surrogate markers as trial endpoints, pivotal trials had larger treatment effects than postapproval trials. On average, treatment effects were 50% higher (more beneficial) in the pivotal than the postapproval trials (ROR 1.5; 95% confidence interval CI 1.01-2.23). For 17 (54.8%) of the 31 matches using the same continuous surrogate markers as trial endpoints, pivotal trials had larger treatment effects than the postapproval trials. On average, there was no difference in treatment effects between pivotal and postapproval trials (dSMDs 0.01; 95% CI -0.15-0.16).

Conclusions: Many postapproval drug trials are not directly comparable to previously published pivotal trials, particularly with respect to endpoint selection. Although treatment effects from pivotal trials supporting FDA approval of novel therapeutics based on non-continuous surrogate markers of disease are often larger than those observed among postapproval trials using surrogate markers as trial endpoints, there is no evidence of difference between pivotal and postapproval trials using continuous surrogate markers.

Keywords: Lifecycle evaluation; Outcomes; Regulation; Surrogate markers; U.S. Food and Drug Administration (FDA).

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

This study used publicly available information and did not require ethics approval from the Yale University School of Medicine Human Research Protection Program.

Consent for publication

Not applicable.

Competing interests

OC is funded by a postdoctoral research fellowship at the University of Exeter Medical School. In the past 36 months, JDW, GSG, and JSR received research support through Yale from the Laura and John Arnold Foundation to support the Collaboration on Research Integrity and Transparency at Yale and JDW received research support through the Meta Research Innovation Center at Stanford (METRICS) from the Laura and John Arnold Foundation. HMK and JSR received research support through Yale from Johnson and Johnson to develop methods of clinical trial data sharing, from Medtronic Inc. and the FDA to develop methods for the postmarket surveillance of medical devices, and from the Centers of Medicare and Medicaid Services to develop and maintain performance measures that are used for public reporting. HMK received compensation as a member of the Scientific Advisory Board for United Healthcare and JSR received research support through Yale from the FDA to establish a Center for Excellence in Regulatory Science and Innovation at Yale University and the Mayo Clinic, from the Blue Cross Blue Shield Association to understand medical technology evaluation better, and from the Agency for Healthcare Research and Quality (R01HS022882).

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig 1
Fig 1
Sample construction for pivotal trials using surrogate markers with matched postapproval trials using surrogate markers
Fig. 2
Fig. 2
Individual and summary odds ratios from pivotal and postapproval trials fulfilling at least two matching criteria and reporting non-continuous endpoints
Fig. 3
Fig. 3
Ratios of odds ratios comparing pivotal and postapproval trials fulfilling at least two matching criteria and reporting non-continuous endpoints
Fig. 4
Fig. 4
Individual and summary standardized mean differences from pivotal and postapproval trials fulfilling at least two matching criteria and reporting continuous endpoints
Fig. 5
Fig. 5
Differences between standardized mean differences comparing pivotal and postapproval trials fulfilling at least two matching criteria and reporting continuous endpoints
Fig. 6
Fig. 6
Comparison of surrogate-based effect estimates from pivotal trials and post approval trials. One outlier not show on graph
See this image and copyright information in PMC

References

    1. U.S. Food and Drug Administration (FDA). Guidance for industry: providing clinical evidence of effectiveness for human drugs and biological products. https://www.fda.gov/ohrms/dockets/ac/06/briefing/2006-4227B1-02-02-FDA-A.... Accessed 11 Nov 2017.
    1. Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005–2012. JAMA. 2014;311(4):368–377. doi: 10.1001/jama.2013.282034. - DOI - PMC - PubMed
    1. Lassere MN. The Biomarker-Surrogacy Evaluation Schema: a review of the biomarker-surrogate literature and a proposal for a criterion-based, quantitative, multidimensional hierarchical levels of evidence schema for evaluating the status of biomarkers as surrogate endpoints. Stat Methods Med Res. 2008;17(3):303–340. doi: 10.1177/0962280207082719. - DOI - PubMed
    1. Wittes J, Lakatos E, Probstfield J. Surrogate endpoints in clinical trials: cardiovascular diseases. Stat Med. 1989;8(4):415–425. doi: 10.1002/sim.4780080405. - DOI - PubMed
    1. Ciani O, Buyse M, Drummond M, Rasi G, Saad ED, Taylor RS. Time to review the role of surrogate end points in health policy: state of the art and the way forward. Value Health. 2017;20(3):487–495. doi: 10.1016/j.jval.2016.10.011. - DOI - PubMed

Publication types

LinkOut - more resources