Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Oct 1;31(10):2294-2303.
doi: 10.1093/jamia/ocae186.

Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency

Cathy Shyr  1 , Randall W Grout  2   3 , Nan Kennedy  4 , Yasemin Akdas  5 , Maeve Tischbein  4 , Joshua Milford  4 , Jason Tan  4 , Kaysi Quarles  4 , Terri L Edwards  4 , Laurie L Novak  1 , Jules White  6 , Consuelo H Wilkins  7 , Paul A Harris  1   8   9
Affiliations

Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency

Cathy Shyr et al. J Am Med Inform Assoc. .

Abstract

Objective: Returning aggregate study results is an important ethical responsibility to promote trust and inform decision making, but the practice of providing results to a lay audience is not widely adopted. Barriers include significant cost and time required to develop lay summaries and scarce infrastructure necessary for returning them to the public. Our study aims to generate, evaluate, and implement ChatGPT 4 lay summaries of scientific abstracts on a national clinical study recruitment platform, ResearchMatch, to facilitate timely and cost-effective return of study results at scale.

Materials and methods: We engineered prompts to summarize abstracts at a literacy level accessible to the public, prioritizing succinctness, clarity, and practical relevance. Researchers and volunteers assessed ChatGPT-generated lay summaries across five dimensions: accuracy, relevance, accessibility, transparency, and harmfulness. We used precision analysis and adaptive random sampling to determine the optimal number of summaries for evaluation, ensuring high statistical precision.

Results: ChatGPT achieved 95.9% (95% CI, 92.1-97.9) accuracy and 96.2% (92.4-98.1) relevance across 192 summary sentences from 33 abstracts based on researcher review. 85.3% (69.9-93.6) of 34 volunteers perceived ChatGPT-generated summaries as more accessible and 73.5% (56.9-85.4) more transparent than the original abstract. None of the summaries were deemed harmful. We expanded ResearchMatch's technical infrastructure to automatically generate and display lay summaries for over 750 published studies that resulted from the platform's recruitment mechanism.

Discussion and conclusion: Implementing AI-generated lay summaries on ResearchMatch demonstrates the potential of a scalable framework generalizable to broader platforms for enhancing research accessibility and transparency.

Keywords: ResearchMatch; artificial intelligence; large language model; return of study results; text summarization.

PubMed Disclaimer

Conflict of interest statement

None.

Figures

Figure 1.
Figure 1.
Overview of study.
Figure 2.
Figure 2.
Overview of chain-of-thought prompting.
Figure 3.
Figure 3.
Example of abstract and lay summary on ResearchMatch.
See this image and copyright information in PMC

Comment in

References

    1. Getz K, Farides-Mitchell J. Assessing the adoption of clinical trial results summary disclosure to patients and the public. Expert Rev Clin Pharmacol. 2019;12(7):573-578. 10.1080/17512433.2019.1615441 - DOI - PubMed
    1. Long CR, Purvis RS, Flood-Grady E, et al. Health researchers’ experiences, perceptions and barriers related to sharing study results with participants. Health Res Policy Syst. 2019;17(1):25. 10.1186/s12961-019-0422-5 - DOI - PMC - PubMed
    1. Patient-Centered Outcomes Research Institute. Returning study results to participants: an important responsibility. Accessed April 22, 2024. https://www.pcori.org/research/about-our-research/returning-study-result...
    1. Rigby H, Fernandez CV. Providing research results to study participants: support versus practice of researchers presenting at the American Society of Hematology annual meeting. Blood. 2005;106(4):1199-1202. 10.1182/blood-2005-02-0556 - DOI - PubMed
    1. Purvis RS, Abraham TH, Long CR, Stewart MK, Warmack TS, McElfish PA. Qualitative study of participants’ perceptions and preferences regarding research dissemination. AJOB Empir Bioeth. 2017;8(2):69-74. 10.1080/23294515.2017.1310146 - DOI - PMC - PubMed