Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep 29;15(9):e46213.
doi: 10.7759/cureus.46213. eCollection 2023 Sep.

Utilizing ChatGPT-4 for Providing Information on Periodontal Disease to Patients: A DISCERN Quality Analysis

Raif Alan  1 , Betül Melek Alan  2
Affiliations

Utilizing ChatGPT-4 for Providing Information on Periodontal Disease to Patients: A DISCERN Quality Analysis

Raif Alan et al. Cureus. .

Abstract

Background: Due to their ability to mimic human responses, anthropomorphic entities such as ChatGPT have a higher likelihood of gaining people's trust. This study aimed to evaluate the quality of information generated by ChatGPT-4, as an artificial intelligence (AI) chatbot, on periodontal disease (PD) using the DISCERN instrument.

Methods: Using Google Bard, the topics related to PD that had the highest search volume according to Google Trends were identified. An interactive dialogue was created by placing the topics in the standard question pattern. As a patient with PD, detailed information was requested from ChatGPT-4 regarding the relevant topics. The 'regenerate response' feature was not employed, and the initial response generated by ChatGPT-4 was carefully considered for each topic as new prompts in the form of questions were entered. The response to each question was independently assessed and rated by two experienced raters using the DISCERN instrument.

Results: Based on the total DISCERN scores, the qualities of the responses generated by ChatGPT-4 were 'good', except for the two responses that rater-2 scored as 'fair'. It was also observed that the 'treatment choices' section of both raters had significantly fewer scores than the other sections. In both weighted kappa and Krippendorff alpha measures, the strength of agreement varied from 'substantial' to 'almost-perfect', and the correlation between values was statistically significant.

Conclusion: Despite some limitations in providing complete treatment choice information according to the DISCERN instrument, it is considered valuable for PD patients seeking information, as it consistently offered accurate guidance in the majority of responses.

Keywords: artificial intelligence; chatbot; health information management; oral health; periodontal disease.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Figure 1
Figure 1. The brief process of the study method.
Figure 2
Figure 2. The performance of ChatGPT-4-generated information evaluated by independent raters using the DISCERN instrument.
Figure 3
Figure 3. Inter-rater and intra-rater comparison of DISCERN sections.
*: p < 0.001
Figure 4
Figure 4. Correlation between the reliability measures (ρ = 0.687; p < 0.05).
See this image and copyright information in PMC

References

    1. Periodontal infectogenomics. Kaur G, Grover V, Bhaskar N, Kaur RK, Jain A. Inflamm Regen. 2018;38:8. - PMC - PubMed
    1. Impact of the global burden of periodontal diseases on health, nutrition and wellbeing of mankind: a call for global action. Tonetti MS, Jepsen S, Jin L, Otomo-Corgel J. J Clin Periodontol. 2017;44:456–462. - PubMed
    1. Global burden of severe periodontitis in 1990-2010: a systematic review and meta-regression. Kassebaum NJ, Bernabé E, Dahiya M, Bhandari B, Murray CJ, Marcenes W. J Dent Res. 2014;93:1045–1053. - PMC - PubMed
    1. Compliance, concordance, adherence. Aronson JK. Br J Clin Pharmacol. 2007;63:383–384. - PMC - PubMed
    1. Supportive periodontal therapy (SPT) for maintaining the dentition in adults treated for periodontitis. Manresa C, Sanz-Miralles EC, Twigg J, Bravo M. Cochrane Database Syst Rev. 2018;1:0. - PMC - PubMed

LinkOut - more resources