Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jul;13(4):675-684.
doi: 10.1111/cts.12764. Epub 2020 Apr 10.

Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Zhe He  1 , Xiang Tang  2 , Xi Yang  3 , Yi Guo  3 , Thomas J George  4 , Neil Charness  5 , Kelsa Bartley Quan Hem  6 , William Hogan  3 , Jiang Bian  3
Affiliations

Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Zhe He et al. Clin Transl Sci. 2020 Jul.

Abstract

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long-standing concern when applying trial results to real-world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real-world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.

PubMed Disclaimer

Conflict of interest statement

The authors declared no competing interests for this work.

Figures

Figure 1
Figure 1
The PRISMA flow diagram of the review. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta‐Analyses.
Figure 2
Figure 2
The numbers of generalizability assessment studies from 1985 to April 2019.
Figure 3
Figure 3
A taxonomy of generalizability assessment methods. Boxes (a) and (b) list the different types of populations compared in a priori and a posteriori generalizability assessment articles, respectively.
Figure 4
Figure 4
The yearly trend of generalizability assessment publications by methods in terms of data availability.
Figure 5
Figure 5
Trends of the data source types used for profiling the target populations.
See this image and copyright information in PMC

References

    1. From the NIH Director: the importance of clinical trials <http://www.nlm.nih.gov/medlineplus/magazine/issues/summer11/articles/sum...>. Accessed April 1, 2019.
    1. Sedgwick, P. External and internal validity in clinical trials. BMJ. 344, e1004 (2012).
    1. Beaver, J.A. , Ison, G. & Pazdur, R. Reevaluating eligibility criteria—balancing patient protection and participation in oncology trials. N. Engl. J. Med . 376, 1504–1505 (2017). - PubMed
    1. Shields, K.E. & Lyerly, A.D. Exclusion of pregnant women from industry‐sponsored clinical trials. Obstet. Gynecol. 122, 1077–1081 (2013). - PubMed
    1. US Food and Drug Administration . Enhancing the diversity of clinical trial populations—eligibility criteria, enrollment practices, and trial designs guidance for industry <https://www.fda.gov/regulatory‐information/search‐fda‐guidance‐documents...> (2019). Accessed August 5, 2019.

Publication types

MeSH terms