Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2022 Apr;27(2):109-119.
doi: 10.1136/bmjebm-2020-111493. Epub 2020 Dec 9.

Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making

Grammati Sarri  1 , Elisabetta Patorno  2 , Hongbo Yuan  3 , Jianfei Jeff Guo  4 , Dimitri Bennett  5 , Xuerong Wen  6 , Andrew R Zullo  7 , Joan Largent  8 , Mary Panaccio  9 , Mugdha Gokhale  10 , Daniela Claudia Moga  11 , M Sanni Ali  12   13   14 , Thomas P A Debray  15   16
Affiliations
Meta-Analysis

Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making

Grammati Sarri et al. BMJ Evid Based Med. 2022 Apr.

Abstract

Introduction: High-quality randomised controlled trials (RCTs) provide the most reliable evidence on the comparative efficacy of new medicines. However, non-randomised studies (NRS) are increasingly recognised as a source of insights into the real-world performance of novel therapeutic products, particularly when traditional RCTs are impractical or lack generalisability. This means there is a growing need for synthesising evidence from RCTs and NRS in healthcare decision making, particularly given recent developments such as innovative study designs, digital technologies and linked databases across countries. Crucially, however, no formal framework exists to guide the integration of these data types.

Objectives and methods: To address this gap, we used a mixed methods approach (review of existing guidance, methodological papers, Delphi survey) to develop guidance for researchers and healthcare decision-makers on when and how to best combine evidence from NRS and RCTs to improve transparency and build confidence in the resulting summary effect estimates.

Results: Our framework comprises seven steps on guiding the integration and interpretation of evidence from NRS and RCTs and we offer recommendations on the most appropriate statistical approaches based on three main analytical scenarios in healthcare decision making (specifically, 'high-bar evidence' when RCTs are the preferred source of evidence, 'medium,' and 'low' when NRS is the main source of inference).

Conclusion: Our framework augments existing guidance on assessing the quality of NRS and their compatibility with RCTs for evidence synthesis, while also highlighting potential challenges in implementing it. This manuscript received endorsement from the International Society for Pharmacoepidemiology.

Keywords: evidence-based practice; health care economics and organisations; health services research.

PubMed Disclaimer

Conflict of interest statement

Competing interests: We have read and understood BMJ Evidence-Based Medicine policy on declaration of interests and declare the following interests: GS is employed by Visible Analytics, Ltd; DB is employed by Takeda; ARZ holds a grant from Sanofi Pasteur (direct to institution); MP owns stocks from Merck, Sanofi, and Johnson & Johnson; MG is employed by GSK; and TD is an advisor to pharma industry.

Figures

Figure 1
Figure 1
International Society for Pharmacoepidemiology (ISPE) CER SIG framework for combining NRS with RCTs. CER, comparative effectiveness research; NRS, non-randomised studies; RCT, randomised controlled trial; SIG, Special Interest Group.
Figure 2
Figure 2
Evidence generation needs in healthcare decision setting and use of non-randomised studies (NRS) with randomised controlled trials (RCTs).
Figure 3
Figure 3
A seven-sStep decision algorithm for the synthesis of non-randomised studies (NRS) and randomised controlled trials (RCTs) in healthcare decision-making (ISPE CER SIG framework). CER, comparative effectiveness research; ISPE, International Society for Pharmacoepidemiology; PICOST, Population, Intervention, Comparison, Outcomes, Study design and Time; SIG, Special Interest Group.
See this image and copyright information in PMC

References

    1. Krause JH, Saver RS. Real-World evidence in the real world: beyond the FDA. Am J Law Med 2018;44:161–79. 10.1177/0098858818789423 - DOI - PubMed
    1. Duke University Margolis Center for Health Policy . A framework for regulatory use of real-world evidence, 2017. Available: https://healthpolicy.duke.edu/sites/default/files/atoms/files/rwe_white_...
    1. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCEPP) . ENCEPP Considerations on the Definition of Non-Interventional Trials under the Current Legislative Framework (“Clinical Trials Directive” 2001/20/Ec), 2011. Available: http://www.encepp.eu/publications/documents/ENCePPinterpretationofnonint...
    1. Eichler H-G, Abadie E, Breckenridge A, et al. Bridging the efficacy-effectiveness gap: a regulator's perspective on addressing variability of drug response. Nat Rev Drug Discov 2011;10:495–506. 10.1038/nrd3501 - DOI - PubMed
    1. Canadian Agency for Drugs and Technologies in Health . Use of real-world evidence in single drug technology assessment processes by health technology assessment and regulatory organizations. CADTH, 2018.