Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Oct 9;15(10):e46736.
doi: 10.7759/cureus.46736. eCollection 2023 Oct.

Reliability and Usefulness of ChatGPT for Inflammatory Bowel Diseases: An Analysis for Patients and Healthcare Professionals

Rasim Eren Cankurtaran  1 , Yunus Halil Polat  2 , Neslihan Gunes Aydemir  3 , Ebru Umay  4 , Oyku Tayfur Yurekli  5
Affiliations

Reliability and Usefulness of ChatGPT for Inflammatory Bowel Diseases: An Analysis for Patients and Healthcare Professionals

Rasim Eren Cankurtaran et al. Cureus. .

Abstract

Aim: We aimed to evaluate the performance of Chat Generative Pre-trained Transformer (ChatGPT) within the context of inflammatory bowel disease (IBD), which is expected to become an increasingly significant health issue in the future. In addition, the objective of the study was to assess whether ChatGPT serves as a reliable and useful resource for both patients and healthcare professionals.

Methods: For this study, 20 specific questions were identified for the two main components of IBD, which are Crohn's disease (CD) and ulcerative colitis (UC). The questions were divided into two sets: one set contained questions directed at healthcare professionals while the second set contained questions directed toward patients. The responses were evaluated with seven-point Likert-type reliability and usefulness scales.

Results: The distribution of the reliability and utility scores was calculated into four groups (two diseases and two question sources) by averaging the mean scores from both raters. The highest scores in both reliability and usefulness were obtained from professional sources (5.00± 1.21 and 5.15±1.08, respectively). The ranking in terms of reliability and usefulness, respectively, was as follows: CD questions (4.70±1.26 and 4.75±1.06) and UC questions (4.40±1.21 and 4.55±1.31). The reliability scores of the answers for the professionals were significantly higher than those for the patients (both raters, p=0.032). Conclusion: Despite its capacity for reliability and usefulness in the context of IBD, ChatGPT still has some limitations and deficiencies. The correction of ChatGPT's deficiencies and its enhancement by developers with more detailed and up-to-date information could make it a significant source of information for both patients and medical professionals.

Keywords: artificial intelligence (ai); chatgpt; crohn’s disease (cd); healthcare research; inflammatory bowel diseases (ibd); large language model; ulcerative colitis (uc).

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Figure 1
Figure 1. Distribution of reliability and usefulness scores, which were calculated by averaging the mean scores from both raters.
UC: ulcerative colitis; CD: Crohn's disease; R: reliability; U: usefulness; Prof: professionals
See this image and copyright information in PMC

References

    1. Worldwide incidence and prevalence of inflammatory bowel disease in the 21st century: a systematic review of population-based studies. Ng SC, Shi HY, Hamidi N, et al. Lancet. 2017;390:2769–2778. - PubMed
    1. British Society of Gastroenterology consensus guidelines on the management of inflammatory bowel disease in adults. Lamb CA, Kennedy NA, Raine T, et al. Gut. 2019;68:0. - PMC - PubMed
    1. Introduction to artificial intelligence in medicine. Mintz Y, Brodie R. Minim Invasive Ther Allied Technol. 2019;28:73–81. - PubMed
    1. Physicians’ perceptions of chatbots in health care: cross-sectional web-based survey. Palanica A, Flaschner P, Thommandram A, Li M, Fossat Y. J Med Internet Res. 2019;21:0. - PMC - PubMed
    1. The effectiveness of artificial intelligence conversational agents in health care: systematic review. Milne-Ives M, de Cock C, Lim E, et al. J Med Internet Res. 2020;22:0. - PMC - PubMed

LinkOut - more resources