Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 2:10:20552076231224603.
doi: 10.1177/20552076231224603. eCollection 2024 Jan-Dec.

Evaluation of information provided to patients by ChatGPT about chronic diseases in Spanish language

María Juliana Soto-Chávez  1 , Marlon Mauricio Bustos  1   2 , Daniel G Fernández-Ávila  1   3 , Oscar Mauricio Muñoz  1   2
Affiliations

Evaluation of information provided to patients by ChatGPT about chronic diseases in Spanish language

María Juliana Soto-Chávez et al. Digit Health. .

Abstract

Introduction: Artificial intelligence has presented exponential growth in medicine. The ChatGPT language model has been highlighted as a possible source of patient information. This study evaluates the reliability and readability of ChatGPT-generated patient information on chronic diseases in Spanish.

Methods: Questions frequently asked by patients on the internet about diabetes mellitus, heart failure, rheumatoid arthritis (RA), chronic kidney disease (CKD), and systemic lupus erythematosus (SLE) were submitted to ChatGPT. Reliability was assessed by rating responses as (1) comprehensive, (2) correct but inadequate, (3) some correct and some incorrect, (4) completely incorrect, and divided between "good" (1 and 2) and "bad" (3 and 4). Readability was evaluated with the adapted Flesch and Szigriszt formulas.

Results: And 71.67% of the answers were "good," with none qualified as "completely incorrect." Better reliability was observed in questions on diabetes and RA versus heart failure (p = 0.02). In readability, responses were "moderately difficult" (54.73, interquartile range (IQR) 51.59-58.58), with better results for CKD (median 56.1, IQR 53.5-59.1) and RA (56.4, IQR 53.7-60.7), than for heart failure responses (median 50.6, IQR 46.3-53.8).

Conclusion: Our study suggests that the ChatGPT tool can be a reliable source of information in spanish for patients with chronic diseases with different reliability for some of them, however, it needs to improve the readability of its answers to be recommended as a useful tool for patients.

Keywords: Artificial intelligence; ChatGPT; chronic diseases; readability; reliability.

PubMed Disclaimer

Conflict of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.
Figure 1.
Qualification of responses generated by ChatGPT about chronic diseases.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. IBM. What is artificial intelligence? [Internet]. 2023. Available from: https://www.ibm.com/topics/artificial-intelligence
    1. Zaar O, Larson A, Polesie S, et al. Evaluation of the diagnostic accuracy of an online artificial intelligence application for skin disease diagnosis. Acta Derm Venereol 2020; 100: 1–6. - PMC - PubMed
    1. Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Futur Healthc J 2019; 6: 94–98. - PMC - PubMed
    1. Fox S. Online Health Search 2006 [Internet]. 2006 [cited 2022 Feb 26]. Available from: https://www.pewresearch.org/internet/2006/10/29/online-health-search-2006/
    1. Salah M. Chatting with ChatGPT: decoding the mind of Chatbot users and unveiling the intricate connections between user perception, trust and stereotype perception on self-esteem and psychological well-being. Curr Psychol 2023: 1–26. Preprint. 10.21203/rs.3.rs-2610655/v2 - DOI