Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jun;134(6):2757-2761.
doi: 10.1002/lary.31243. Epub 2023 Dec 21.

Exploring the Role of Artificial Intelligence Chatbots in Preoperative Counseling for Head and Neck Cancer Surgery

Jason C Lee  1 , Chelsea S Hamill  1 , Yelizaveta Shnayder  1 , Erin Buczek  1 , Kiran Kakarala  1 , Andrés M Bur  1
Affiliations

Exploring the Role of Artificial Intelligence Chatbots in Preoperative Counseling for Head and Neck Cancer Surgery

Jason C Lee et al. Laryngoscope. 2024 Jun.

Abstract

Objective: To evaluate the potential use of artificial intelligence (AI) chatbots, such as ChatGPT, in preoperative counseling for patients undergoing head and neck cancer surgery.

Study design: Cross-Sectional Survey Study.

Setting: Single institution tertiary care center.

Methods: ChatGPT was used to generate presurgical educational information including indications, risks, and recovery time for five common head and neck surgeries. Chatbot-generated information was compared with information gathered from a simple browser search (first publicly available website excluding scholarly articles). The accuracy of the information, readability, thoroughness, and number of errors were compared by five experienced head and neck surgeons in a blinded fashion. Each surgeon then chose a preference between the two information sources for each surgery.

Results: With the exception of total word count, ChatGPT-generated pre-surgical information has similar readability, content of knowledge, accuracy, thoroughness, and numbers of medical errors when compared to publicly available websites. Additionally, ChatGPT was preferred 48% of the time by experienced head and neck surgeons.

Conclusion: Head and neck surgeons rated ChatGPT-generated and readily available online educational materials similarly. Further refinement in AI technology may soon open more avenues for patient counseling. Future investigations into the medical safety of AI counseling and exploring patients' perspectives would be of strong interest.

Level of evidence: N/A. Laryngoscope, 134:2757-2761, 2024.

Keywords: ChatGPT; artificial intelligence; chatbot; head and neck surgery; patient counseling.

PubMed Disclaimer

Conflict of interest statement

The authors have no funding, financial relationships, or conflicts of interest to disclose.

Figures

Fig. 1.
Fig. 1.
Comparison of ChatGPT-generated texts to web browsing from freely available websites. (A) ChatGPT-generated tests are significantly wordier. (B and C) addition of prompt modifiers, such as “explain… using 6th grade reading comprehension” resulted in improved readability. (D) ChatGPT-generated texts have similar content of knowledge regardless of prompt modifiers. ns, not significant with p > 0.05; * p ≤ 0.05; ** p ≤ 0.01; ***, p ≤ 0.001.
Fig. 2.
Fig. 2.
Assessment of medical accuracy, thoroughness, and number of medical errors by head and neck surgeons. ChatGPT-generated texts scored similarly to publicly available websites in terms of accuracy (A), thoroughness (B), and number of medical errors (C). ns, not significant with p > 0.05; * p ≤ 0.05; ** p ≤ 0.01; *** p ≤ 0.001.
See this image and copyright information in PMC

References

    1. Husson O, Mols F, van de Poll-Franse LV. The relation between information provision and health-related quality of life, anxiety and depression among cancer survivors: a systematic review. Ann Oncol. 2011;22(4):761–772. 10.1093/annonc/mdq413. - DOI - PMC - PubMed
    1. Harrison JD, Young JM, Price MA, Butow PN, Solomon MJ. What are the unmet supportive care needs of people with cancer? A systematic review. Support Care Cancer. 2009;17(8):1117–1128. 10.1007/s00520-009-0615-5. - DOI - PubMed
    1. Llewellyn CD, McGurk M, Weinman J. How satisfied are head and neck cancer (HNC) patients with the information they receive pre-treatment? Results from the satisfaction with cancer information profile (SCIP). Oral Oncol. 2006;42(7):726–734. 10.1016/j.oraloncology.2005.11.013. - DOI - PubMed
    1. Nallani R, Smith JB, Penn JP, et al. Decision regret 3 and 6 months after treatment for head and neck cancer: observational study of associations with clinicodemographics, anxiety, and quality of life. Head Neck. 2022;44(1):59–70. 10.1002/hed.26911. - DOI - PubMed
    1. Amanian A, Heffernan A, Ishii M, Creighton FX, Thamboo A. The evolution and application of artificial intelligence in rhinology: a state of the art review. Otolaryngol Head Neck Surg. 2022;169(1):21–30. 10.1177/01945998221110076. - DOI - PMC - PubMed