Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Mar 25;333(12):1084-1087.
doi: 10.1001/jama.2024.28047.

Manual vs AI-Assisted Prescreening for Trial Eligibility Using Large Language Models-A Randomized Clinical Trial

Ozan Unlu  1   2   3 , Matthew Varugheese  1   4   5 , Jiyeon Shin  1   4   5 , Samantha M Subramaniam  1   2 , David Walter Jacques Stein  2   3 , John J St Laurent  1   4 , Charlotte J Mailly  1   4   5 , Marian J McPartlin  1   6 , Fei Wang  1   4   5 , Michael F Oates  1   5   6 , Christopher P Cannon  1   2   3 , Benjamin M Scirica  1   2   3 , Kavishwar B Wagholikar  1   7 , Samuel J Aronson  1   4   5 , Alexander J Blood  1   2   3
Affiliations

Manual vs AI-Assisted Prescreening for Trial Eligibility Using Large Language Models-A Randomized Clinical Trial

Ozan Unlu et al. JAMA. .
No abstract available

Plain language summary

This randomized clinical trial compares the efficiency of prescreening patients with heart failure using an artificial intelligence (AI) large language model for inclusion in a clinical trial vs manual prescreening.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Unlu reported receiving funding from the National Heart, Lung, and Blood Institute (award T32HL007604). Dr Blood, Messrs Varugheese, Wang, Oates, and Aronson, and Mss Shin, Subramaniam, and Mailly reported receiving grants from Boehringer Ingelheim, Better Therapeutics, Foresite Labs, Milestone Pharmaceutical, Novo Nordisk, and Pfizer. Ms McPartlin reported receiving grants from Boehringer Ingelheim, Pfizer, and Novo Nordisk. Dr Cannon reported receiving grants from Amgen, Better Therapeutics, Boehringer Ingelheim, and Novo Nordisk and receiving personal fees from Amryt/Chiesi, Amgen, Ascendia, Biogen, Boehringer Ingelheim, Bristol Myers Squibb, CSL Behring, Genomadix, Lilly, Janssen, Lexicon, Milestone, Pfizer, and Rhoshan. Dr Scirica reported receiving grants from Boehringer Ingelheim, Amgen, Milestone Pharmaceutical, Merck, Novo Nordisk, Verve, and Pfizer, receiving personal fees from AbbVie, AstraZeneca, Bayer, Hanmi, Lexeo, Novo Nordisk, Verve, and Boerhinger Ingelheim, and having equity in Health at Scale, Aboretun LifeSciences, and Doximity. Mr Aronson also reported receiving personal fees from Nest Genomics and Harvard Medical School. Dr Blood also reported receiving grants from General Electric Health, receiving personal fees from Alnylam, Milestone Therapeutics, NODE Health, Walgreens Health, Medscape, Color Health, Corcept Therapeutics, Nference Inc, Withings, and Arsenal Capital Partners and having equity in Knownwell Health, Porter Health, and Signum Technologies. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Diagram of Study Flow
AI indicates artificial intelligence; LVEF, left ventricular ejection fraction. aEligibility criteria were determined based on structured data queries. bStudy staff were allocated equal time to each screening method and crossed over between groups an equal number of times throughout the study period.
Figure 2.
Figure 2.. Cumulative Incidence of Eligibility Determination and Enrollment
The follow-up period for each participant varied between 1 and 4 months with a mean time from randomization to study end of 81.3 days (SD, 25.8 days). The cumulative incidence estimates represent the probability of eligibility (eligibility state percentage occupancy) at each time point, accounting for competing risks rather than the proportion of patients who ultimately end in each state by the conclusion of the study. Therefore, the plateau of a curve reflects the probability of experiencing the event while considering competing risks and does not directly equate to the final observed proportions of patients in each state. The bands around the lines in part A represent 95% CIs. For the patients who could complete screening, the proportion of eligible patients was similar between the groups (20.8% [458/2205] for the artificial intelligence [AI] screening group and 21.1% [284/1347] for the manual screening group; P = .86). For patients who completed eligibility screening, more than 99% of patients were identified within 15 days in the AI group and within 50 days in the manual group. At the end of the trial, the AI group had 37 patients remaining to be screened vs 887 patients in the manual group.
See this image and copyright information in PMC

References

    1. Deloitte . Unleash AI’s potential: measuring the return from pharmaceutical innovation. Accessed November 25, 2024. https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/arti...
    1. Moore TJ, Heyward J, Anderson G, Alexander GC. Variation in the estimated costs of pivotal clinical benefit trials supporting the US approval of new therapeutic agents, 2015-2017: a cross-sectional study. BMJ Open. 2020;10(6):e038863. doi: 10.1136/bmjopen-2020-038863 - DOI - PMC - PubMed
    1. Davidson MH, Hsieh A, Dicklin MR, Maki KC. The imperative to enhance cost-effectiveness for cardiovascular therapeutic development. JACC Basic Transl Sci. 2024;9(8):1029-1040. doi: 10.1016/j.jacbts.2023.12.005 - DOI - PMC - PubMed
    1. Deloitte . Intelligent clinical trials. Accessed November 25, 2024. https://www2.deloitte.com/content/dam/insights/us/articles/22934_intelli...
    1. Unlu O, Shin J, Mailly CJ, et al. Retrieval-augmented generation–enabled GPT-4 for clinical trial screening. NEJM AI. Published online June 17, 2024. doi: 10.1056/AIoa2400181 - DOI

LinkOut - more resources