Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 3;48(1):38.
doi: 10.1007/s10916-024-02056-0.

Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis

Mehmet Fatih Şahin  1 , Hüseyin Ateş  2 , Anıl Keleş  2 , Rıdvan Özcan  3 , Çağrı Doğan  2 , Murat Akgül  2 , Cenk Murat Yazıcı  2
Affiliations

Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis

Mehmet Fatih Şahin et al. J Med Syst. .

Abstract

The aim of the study is to evaluate and compare the quality and readability of responses generated by five different artificial intelligence (AI) chatbots-ChatGPT, Bard, Bing, Ernie, and Copilot-to the top searched queries of erectile dysfunction (ED). Google Trends was used to identify ED-related relevant phrases. Each AI chatbot received a specific sequence of 25 frequently searched terms as input. Responses were evaluated using DISCERN, Ensuring Quality Information for Patients (EQIP), and Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) metrics. The top three most frequently searched phrases were "erectile dysfunction cause", "how to erectile dysfunction," and "erectile dysfunction treatment." Zimbabwe, Zambia, and Ghana exhibited the highest level of interest in ED. None of the AI chatbots achieved the necessary degree of readability. However, Bard exhibited significantly higher FKRE and FKGL ratings (p = 0.001), and Copilot achieved better EQIP and DISCERN ratings than the other chatbots (p = 0.001). Bard exhibited the simplest linguistic framework and posed the least challenge in terms of readability and comprehension, and Copilot's text quality on ED was superior to the other chatbots. As new chatbots are introduced, their understandability and text quality increase, providing better guidance to patients.

Keywords: Artificial intelligence; Bard; Bing; ChatGPT; Chatbot; Copilot; Erectile dysfunction; Ernie bot.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Global search interest in ED by region from 2004 to 2023 as determined by Google Trends data (regions with modest search volumes are not included)
Fig. 2
Fig. 2
Global search interest over time, from 2004 to 2023, as determined by Google Trends data
See this image and copyright information in PMC

References

    1. Miller DC, Saigal CS, Litwin MS, et al (2009). The demographic burden of urologic disease in America. Urol Clin North Am; 36:11–27. - PMC - PubMed
    1. Ab Rahman AA, Al-Sadat N, Yun Low W. (2011) Help seeking behaviour among men with erectile dysfunction in primary care setting. J Mens Health.;8: S94–6.
    1. Waling A, Fraser S, Fisher C. Young People and Sources of Sexual Health Information (ARCSHS Monograph Series No. 121). Bundoora, VIC: Australian Research Centre in Sex, Health and Society, La Trobe University 2020.
    1. Russo, G.I., Asmundo, M.G., Durukan, E. et al (2023). Quality and benefits of the erectile dysfunction information on websites, social-media, and applications. Int J Impot Res. - PubMed
    1. Pérez-Soler S, Juarez-Puerta S, Guerra E, de Lara J (2021). Choosing a chatbot development tool. IEEE Software; 38:94–103.