Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jun 2;12(3):qfae036.
doi: 10.1093/sexmed/qfae036. eCollection 2024 Jun.

Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation

Mehmet Fatih Şahin  1 , Anil Keleş  1 , Rıdvan Özcan  2 , Çağrı Doğan  1 , Erdem Can Topkaç  1 , Murat Akgül  1 , Cenk Murat Yazıci  1
Affiliations

Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation

Mehmet Fatih Şahin et al. Sex Med. .

Abstract

Background: Premature ejaculation (PE) is the most prevalent sexual dysfunction in men, and like many diseases and conditions, patients use Internet sources like ChatGPT, which is a popular artificial intelligence-based language model, for queries about this andrological disorder.

Aim: The objective of this research was to evaluate the quality, readability, and understanding of texts produced by ChatGPT in response to frequently requested inquiries on PE.

Methods: In this study we used Google Trends to identify the most frequently searched phrases related to PE. Subsequently, the discovered keywords were methodically entered into ChatGPT, and the resulting replies were assessed for quality using the Ensuring Quality Information for Patients (EQIP) program. The produced texts were assessed for readability using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), and DISCERN metrics.

Outcomes: This investigation has identified substantial concerns about the quality of texts produced by ChatGPT, highlighting severe problems with reading and understanding.

Results: The mean EQIP score for the texts was determined to be 45.93 ± 4.34, while the FRES was 15.8 ± 8.73. Additionally, the FKGL score was computed to be 15.68 ± 1.67 and the DISCERN score was 38.1 ± 3.78. The comparatively low average EQIP and DISCERN scores suggest that improvements are required to increase the quality and dependability of the presented information. In addition, the FKGL scores indicate a significant degree of linguistic intricacy, requiring a level of knowledge comparable to about 14 to 15 years of formal schooling in order to understand. The texts about treatment, which are the most frequently searched items, are more difficult to understand compared to other texts about other categories.

Clinical implications: The results of this research suggest that compared to texts on other topics the PE texts produced by ChatGPT exhibit a higher degree of complexity, which exceeds the recommended reading threshold for effective health communication. Currently, ChatGPT is cannot be considered a substitute for comprehensive medical consultations.

Strengths and limitations: This study is to our knowledge the first reported research investigating the quality and comprehensibility of information generated by ChatGPT in relation to frequently requested queries about PE. The main limitation is that the investigation included only the first 25 popular keywords in English.

Conclusion: ChatGPT is incapable of replacing the need for thorough medical consultations.

Keywords: ChatGPT; Chatbot; artificial intelligence; premature ejaculation.

PubMed Disclaimer

Conflict of interest statement

None of the authors received any type of financial support that could be considered a potential conflict of interest regarding the manuscript or its submission.

Figures

Figure 1
Figure 1
The global search interest by region for premature ejaculation from 2004 to 2023, from Google trends data (low search volume regions included).
Figure 2
Figure 2
The global search interest over time for premature ejaculation from 2004 to 2023 using Google trends data.
See this image and copyright information in PMC

References

    1. Patrick DL, Althof SE, Pryor JL, et al. . Premature ejaculation: an observational study of men and their partners. J Sex Med. 2005;2(3):358–367. 10.1111/j.1743-6109.2005.20353.x - DOI - PubMed
    1. Athanasiadis L. Premature ejaculation: is it a biogenic or a psychogenic disorder? Sex Marital Ther. 1998;13(3):241–255. 10.1080/02674659808406567 - DOI
    1. Martin C, Nolen H, Podolnick J, Wang R. Current and emerging therapies in premature ejaculation: where we are coming from, where we are going. Int J Urol. 2017;24(1):40–50. 10.1111/iju.13202 - DOI - PubMed
    1. Gul M, Kaynar M. Evaluating the content and quality of information about premature ejaculation on the internet: what are men being exposed to? Andrologia. 2017;49(2):e12612. 10.1111/and.12612 - DOI - PubMed
    1. Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Language Understanding by Generative Pre-Training. San Francisco: OpenAI; 2018. https://scholar.google.com/scholar?q=Improving+Language+Understanding+by...