Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul;26(4):746-762.
doi: 10.1177/15248399241285060. Epub 2024 Oct 11.

Suitability of ChatGPT as a Source of Patient Information for Screening Mammography

Kelly Spuur  1 , Geoff Currie  1 , Dana Al-Mousa  1 , Ruth Pape  1
Affiliations

Suitability of ChatGPT as a Source of Patient Information for Screening Mammography

Kelly Spuur et al. Health Promot Pract. 2025 Jul.

Abstract

ChatGPT3.5 and ChatGPT4 were released publicly in late November 2022 and March 2023, respectively, and have emerged as convenient sources of patient health education and information, including for screening mammography. ChatGPT4 offers enhanced capabilities; however, it is only available by paid subscription. The purported benefits of ChatGPT for health education need to be objectively evaluated. To assess performance differences, ChatGPT3.5 and GPT4 were used between 13 April and 29 May 2023 to generate breast screening patient information sheets, which were evaluated using the Patient Education Materials Assessment Tool for printed materials (PEMAT-P) and the CDC Clear Communication Index (CDC Index) Score Sheet; and benchmarked against gold standard content in BreastScreen NSW's patient information sheet. Mean scores were reported for comparison. GPT3.5 provided the appropriate tone and currency of information but lacked accuracy, omitting key insights: PEMAT-P understandability 68.0% (SD = 6.56) and actionability 36.7% (SD=20.4); CDC Index 58.8% (SD = 15.3). GPT4 was deemed superior to GPT3.5 but included several key omissions: PEMAT-P understandability 75.0% (SD = 17) and actionability 53.3% (SD = 11.54); CDC Index 66.0% (SD = 4.1). Both ChatGPT versions exhibited poor understandability and actionability and were unclear in their messaging. Those with poor health literacy will not benefit from accessing current versions of ChatGPT and may be further disadvantaged if they do not have access to a paid subscription. ChatGPT is evidenced to be an unreliable and inaccurate source of information concerning breast screening that may undermine participation and risk increased morbidity and mortality from breast cancer. ChatGPT may increase the demand on health care educators to rectify misinformation.

Keywords: ChatGPT; actionability; artificial intelligence; breast screening; generative AI; language model; mammography; patient education; readability; understandability.

PubMed Disclaimer

Figures

Figure 1
Figure 1
GPT3.5 Generated Patient Information Sheet
Figure 2
Figure 2
GPT4 Generated Patient Information Sheet
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Australian Institute of Health and Welfare. (2021). BreastScreen Australia monitoring report 2021. 10.25816/btjk-3q46 - DOI
    1. Ayoub N. F., Lee Y. J., Grimm D., Divi V. (2024). Head-to-head comparison of ChatGPT versus Google search for medical knowledge acquisition. Otolaryngology–Head and Neck Surgery, 170(6), 1484–1491. 10.1002/ohn.465 - DOI - PubMed
    1. Azer S. A., Al Olayan T. I., AlGhamdi M. A., AlSanea M. A. (2017). Inflammatory bowel disease: An evaluation of health information on the internet. World Journal of Gastroenterology, 23(9), 1676–1696. 10.3748/wjg.v23.i9.1676 - DOI - PMC - PubMed
    1. Baur C., Prue C. (2014). The CDC clear communication index is a new evidence-based tool to prepare and review health information. Health Promotion Practice, 15(5), 629–637. 10.1177/1524839914538969 - DOI - PubMed
    1. Biswas S. (2023). ChatGPT and the future of medical writing. Radiology, 307(2), Article e223312. 10.1148/radiol.223312 - DOI - PubMed