Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Oct 2;2(10):e1912869.
doi: 10.1001/jamanetworkopen.2019.12869.

Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence

Victoria L Bartlett  1 , Sanket S Dhruva  2   3 , Nilay D Shah  4 , Patrick Ryan  5   6 , Joseph S Ross  7   8   9
Affiliations

Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence

Victoria L Bartlett et al. JAMA Netw Open. .

Abstract

Importance: Although randomized clinical trials are considered to be the criterion standard for generating clinical evidence, the use of real-world evidence to evaluate the efficacy and safety of medical interventions is gaining interest. Whether observational data can be used to address the same clinical questions being answered by traditional clinical trials is still unclear.

Objective: To identify the number of clinical trials published in high-impact journals in 2017 that could be feasibly replicated using observational data from insurance claims and/or electronic health records (EHRs).

Design, setting, and participants: In this cross-sectional analysis, PubMed was searched to identify all US-based clinical trials, regardless of randomization, published between January 1, 2017, and December 31, 2017, in the top 7 highest-impact general medical journals of 2017. Trials were excluded if they did not involve human participants, did not use end points that represented clinical outcomes among patients, were not characterized as clinical trials, and had no recruitment sites in the United States.

Main outcomes and measures: The primary outcomes were the number and percentage of trials for which the intervention, indication, trial inclusion and exclusion criteria, and primary end points could be ascertained from insurance claims and/or EHR data.

Results: Of the 220 US-based trials analyzed, 33 (15.0%) could be replicated using observational data because their intervention, indication, inclusion and exclusion criteria, and primary end points could be routinely ascertained from insurance claims and/or EHR data. Of the 220 trials, 86 (39.1%) had an intervention that could be ascertained from insurance claims and/or EHR data. Among the 86 trials, 62 (72.1%) had an indication that could be ascertained. Forty-five (72.6%) of 62 trials had at least 80% of inclusion and exclusion criteria data that could be ascertained. Of these 45 studies, 33 (73.3%) had at least 1 primary end point that could be ascertained.

Conclusions and relevance: This study found that only 15% of the US-based clinical trials published in high-impact journals in 2017 could be feasibly replicated through analysis of administrative claims or EHR data. This finding suggests the potential for real-world evidence to complement clinical trials, both by examining the concordance between randomized experiments and observational studies and by comparing the generalizability of the trial population with the real-world population of interest.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Shah reported receiving grants from the US Food and Drug Administration (FDA), Agency for Healthcare Research and Quality (AHRQ), Center for Medicare and Medicaid Innovation (CMMI), National Heart, Lung and Blood Institute of the National Institutes of Health (NHLBI/NIH), National Science Foundation, and Patient Centered Outcomes Research Institute (PCORI) outside the submitted work and receiving research support through Mayo Clinic from the FDA to establish the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program. Dr Ryan reported being an employee of Janssen Research and Development and a shareholder of Johnson & Johnson. Dr Ross reported receiving grants from the FDA, the Centers for Medicare & Medicaid Services, Medtronic Inc, Blue Cross Blue Shield Association, AHRQ, NHLBI/NIH, and Laura and John Arnold Foundation outside the submitted work and receiving research support through Yale University from Johnson & Johnson. No other disclosures were reported.

Figures

Figure.
Figure.. Study Flowchart
Included are reasons that specific clinical trial characteristics could not be reliably ascertained from electronic health record or claims data. FDA indicates US Food and Drug Administration; ICD, International Classification of Diseases; and QOL, quality of life. aBreakdown does not sum to the total because some trials had multiples of these criteria. Breakdowns count the number of trials with a specific characteristic and not the number of individual criteria or end points.
See this image and copyright information in PMC

References

    1. Sherman RE, Anderson SA, Dal Pan GJ, et al. . Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):-. doi:10.1056/NEJMsb1609216 - DOI - PubMed
    1. Institute of Medicine Forum on Drug Discovery, Development, and Translation Transforming Clinical Research in the United States: Challenges and Opportunities: Workshop Summary. Washington, DC:National Academies Press; 2010. - PubMed
    1. Dhruva SS, Ross JS, Desai NR. Real-world evidence: promise and peril for medical product evaluation. P T. 2018;43(8):464-472. - PMC - PubMed
    1. US Food and Drug Administration Use of real-world evidence to support regulatory decision-making for medical devices: guidance for industry and Food and Drug Administration staff. https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance.... Published August 2017. Accessed August 31, 2019.
    1. National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Health Sciences Policy; Forum on Drug Discovery, Development, and Translation. Real-World Evidence Generation and Evaluation of Therapeutics: Proceedings of a Workshop Washington, DC: National Academies Press; 2017. - PubMed

Publication types

MeSH terms