How Efficient Is ChatGPT in Accessing Accurate and Quality Health-Related Information?

Abstract

Background and objective The field of artificial intelligence (AI) is advancing at a rapid pace, impacting all aspects of human life. Chat Generative Pre-trained Transformer (ChatGPT), which represents one of AI's most recent and remarkable achievements, has garnered significant attention and popularity in the academic community. ChatGPT, a language model-based chatbot developed by OpenAI, responds quickly and provides answers to the questions put to it. This chatbot has the ability to gather content from a variety of sources on the internet. However, its success in providing correct information has not yet been comprehensively analyzed. In light of this, this study aimed to engage in a comparative content analysis of health-related information provided by ChatGPT and a few selected websites. Methods We performed a qualitative analysis of data obtained from various information sources by using the DISCERN score and the Journal of the American Medical Association (JAMA) benchmark criteria. In addition, readability levels of the content were measured by using the Flesch-Kincaid grade level, Gunning Fog Index, and Simple Measure of Gobbledygook (SMOG) index. Results Based on our findings, there was no statistically significant difference between the websites and ChatGPT in DISCERN scores. However, the JAMA score was statistically significantly higher for websites. With regard to the Flesch-Kincaid grade level, Gunning Fog Index, and SMOG index values, the data obtained from the websites had higher readability. Conclusion Although AI is starting to play a significant role in our everyday lives, it has yet to surpass traditional methods of accessing information in terms of readability and reliability.

Keywords: chatgpt; discern; health information; jama; website.