Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jun 4:27:e69955.
doi: 10.2196/69955.

Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study

John Will #  1 , Mahin Gupta #  1 , Jonah Zaretsky #  2 , Aliesha Dowlath #  1 , Paul Testa #  1   3 , Jonah Feldman #  1   4
Affiliations

Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study

John Will et al. J Med Internet Res. .

Abstract

Background: Online accessible patient education materials (PEMs) are essential for patient empowerment. However, studies have shown that these materials often exceed the recommended sixth-grade reading level, making them difficult for many patients to understand. Large language models (LLMs) have the potential to simplify PEMs into more readable educational content.

Objective: We sought to evaluate whether 3 LLMs (ChatGPT [OpenAI], Gemini [Google], and Claude [Anthropic PBC]) can optimize the readability of PEMs to the recommended reading level without compromising accuracy.

Methods: This cross-sectional study used 60 randomly selected PEMs available online from 3 websites. We prompted LLMs to simplify the reading level of online PEMs. The primary outcome was the readability of the original online PEMs compared with the LLM-simplified versions. Readability scores were calculated using 4 validated indices Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook Index. Accuracy and understandability were also assessed as balancing measures, with understandability measured using the Patient Education Materials Assessment Tool-Understandability (PEMAT-U).

Results: The original readability scores for the American Heart Association (AHA), American Cancer Society (ACS), and American Stroke Association (ASA) websites were above the recommended sixth-grade level, with mean grade level scores of 10.7,10.0, and 9.6, respectively. After optimization by the LLMs, readability scores significantly improved across all 3 websites when compared with the original text. Compared with the original website, Wilcoxon signed rank test showed ChatGPT improved the readability to 7.6 from 10.1 (P<.001); Gemini, to 6.6 (P<.001); and Claude, to 5.6 (P<.001). Word counts were significantly reduced by all LLMs, with a decrease from a mean range of 410.9-953.9 words to a mean range of 201.9-248.1 words. None of the ChatGPT LLM-simplified PEMs were inaccurate, while 3.3% of Gemini and Claude LLM-simplified PEMs were inaccurate. Baseline understandability scores, as measured by PEMAT-U, were preserved across all LLM-simplified versions.

Conclusions: This cross-sectional study demonstrates that LLMs have the potential to significantly enhance the readability of online PEMs while maintaining accuracy and understandability, making them more accessible to a broader audience. However, variability in model performance and demonstrated inaccuracies underscore the need for human review of LLM output. Further study is needed to explore advanced LLM techniques and models trained for medical content.

Keywords: artificial intelligence; health education; health literacy; patient education; readability.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Mean grade level scores for the original version of the patient education material and each large language model version (N=60).
See this image and copyright information in PMC

References

    1. Jacobs W, Amuta AO, Jeon KC. Health information seeking in the digital age: An analysis of health information seeking behavior among US adults. Cogent Social Sciences. 2017;3(1):1302785. doi: 10.1080/23311886.2017.1302785. https://doi.org/10.1080/23311886.2017.1302785 - DOI
    1. Tan SSL, Goonawardene N. Internet health information seeking and the patient-physician relationship: A systematic review. J Med Internet Res. 2017;19(1):e9. doi: 10.2196/jmir.5729. https://www.jmir.org/2017/1/e9/ v19i1e9 - DOI - PMC - PubMed
    1. Amante D, Hogan T, Pagoto S, English T, Lapane K. Access to care and use of the internet to search for health information: results from the US national health interview survey. J Med Internet Res. 2015;17(4):e106. doi: 10.2196/jmir.4126. https://www.jmir.org/2015/4/e106/ v17i4e106 - DOI - PMC - PubMed
    1. Swire-Thompson B, Lazer D. Public health and online misinformation: Challenges and recommendations. Annu Rev Public Health. 2020;41:433–451. doi: 10.1146/annurev-publhealth-040119-094127. https://www.annualreviews.org/content/journals/10.1146/annurev-publhealt... - DOI - PubMed
    1. Cutilli CC, Bennett IM. Understanding the health literacy of America: results of the national assessment of adult literacy. Orthop Nurs. 2009;28(1):27–33. doi: 10.1097/01.NOR.0000345852.22122.d6. https://europepmc.org/abstract/MED/19190475 00006416-200901000-00011 - DOI - PMC - PubMed

LinkOut - more resources