Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2024 Nov;14(8):2389-2398.
doi: 10.1177/21925682241241241. Epub 2024 Mar 21.

GPT-4 as a Source of Patient Information for Anterior Cervical Discectomy and Fusion: A Comparative Analysis Against Google Web Search

Paul G Mastrokostas  1 , Leonidas E Mastrokostas  2 , Ahmed K Emara  3 , Ian J Wellington  4 , Elizabeth Ginalis  5 , John K Houten  6 , Amrit S Khalsa  7 , Ahmed Saleh  8 , Afshin E Razi  8 , Mitchell K Ng  8
Affiliations
Review

GPT-4 as a Source of Patient Information for Anterior Cervical Discectomy and Fusion: A Comparative Analysis Against Google Web Search

Paul G Mastrokostas et al. Global Spine J. 2024 Nov.

Abstract

Study design: Comparative study.

Objectives: This study aims to compare Google and GPT-4 in terms of (1) question types, (2) response readability, (3) source quality, and (4) numerical response accuracy for the top 10 most frequently asked questions (FAQs) about anterior cervical discectomy and fusion (ACDF).

Methods: "Anterior cervical discectomy and fusion" was searched on Google and GPT-4 on December 18, 2023. Top 10 FAQs were classified according to the Rothwell system. Source quality was evaluated using JAMA benchmark criteria and readability was assessed using Flesch Reading Ease and Flesch-Kincaid grade level. Differences in JAMA scores, Flesch-Kincaid grade level, Flesch Reading Ease, and word count between platforms were analyzed using Student's t-tests. Statistical significance was set at the .05 level.

Results: Frequently asked questions from Google were varied, while GPT-4 focused on technical details and indications/management. GPT-4 showed a higher Flesch-Kincaid grade level (12.96 vs 9.28, P = .003), lower Flesch Reading Ease score (37.07 vs 54.85, P = .005), and higher JAMA scores for source quality (3.333 vs 1.800, P = .016). Numerically, 6 out of 10 responses varied between platforms, with GPT-4 providing broader recovery timelines for ACDF.

Conclusions: This study demonstrates GPT-4's ability to elevate patient education by providing high-quality, diverse information tailored to those with advanced literacy levels. As AI technology evolves, refining these tools for accuracy and user-friendliness remains crucial, catering to patients' varying literacy levels and information needs in spine surgery.

Keywords: GPT-4; Google; anterior cervical discectomy and fusion; artificial intelligence; health literacy; patient education; readability.

PubMed Disclaimer

Conflict of interest statement

Declaration of Conflicting InterestsThe author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Mitchell K. Ng is a paid consultant at Ferghana Partners. For the remaining authors none were declared.

Figures

Figure 1.
Figure 1.
Rothwell’s classification stratified by search engine.
Figure 2.
Figure 2.
Topic classification stratified by search engine.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Number of ChatGPT Users (Dec 2023). Accessed December 22, 2023. https://explodingtopics.com/blog/chatgpt-users
    1. GPT-4 Released: What it Means for the Future of Your Business. Accessed December 22, 2023. https://www.forbes.com/sites/forbesbusinesscouncil/2023/03/28/gpt-4-rele...
    1. Egli A, Egli A. ChatGPT, GPT-4, and other large language models: the next revolution for clinical microbiology? Clin Infect Dis. 2023;77(9):1322-1328. doi:10.1093/CID/CIAD407 - DOI - PMC - PubMed
    1. Brin D, Sorin V, Vaid A, et al. Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep. 2023;13(1):16492. doi:10.1038/S41598-023-43436-9 - DOI - PMC - PubMed
    1. Currie G, Robbie S, Tually P. ChatGPT and patient information in nuclear medicine: GPT-3.5 versus GPT-4. J Nucl Med Technol. 2023;51(4):307. doi:10.2967/JNMT.123.266151 - DOI - PubMed

LinkOut - more resources