Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Aug 9;26(9):107590.
doi: 10.1016/j.isci.2023.107590. eCollection 2023 Sep 15.

A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons

Jiayu Liu  1 , Jiqi Zheng  2 , Xintian Cai  3 , Dongdong Wu  4 , Chengliang Yin  5
Affiliations

A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons

Jiayu Liu et al. iScience. .

Abstract

ChatGPT is an artificial intelligence product developed by OpenAI. This study aims to investigate whether ChatGPT can respond in accordance with evidence-based medicine in neurosurgery. We generated 50 neurosurgical questions covering neurosurgical diseases. Each question was posed three times to GPT-3.5 and GPT-4.0. We also recruited three neurosurgeons with high, middle, and low seniority to respond to questions. The results were analyzed regarding ChatGPT's overall performance score, mean scores by the items' specialty classification, and question type. In conclusion, GPT-3.5's ability to respond in accordance with evidence-based medicine was comparable to that of neurosurgeons with low seniority, and GPT-4.0's ability was comparable to that of neurosurgeons with high seniority. Although ChatGPT is yet to be comparable to a neurosurgeon with high seniority, future upgrades could enhance its performance and abilities.

Keywords: Artificial intelligence applications; Health informatics; Neurology; Neurosurgery.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

None
Graphical abstract
Figure 1
Figure 1
The comparison of ChatGPT and evidence-based neurosurgeons (A–D) Statistical results (A) the accuracy in different groups; (B) comparison of mean scores in different groups; (C) comparison of mean scores according to question type; (D) comparison of mean scores according to specialty classification.
Figure 2
Figure 2
Screenshot of ChatGPT’s answer to a question
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Howard A., Hope W., Gerada A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect. Dis. 2023;23:405–406. doi: 10.1016/S1473-3099(23)00113-5. - DOI - PubMed
    1. Mann D.L. Artificial Intelligence Discusses the Role of Artificial Intelligence in Translational Medicine: A JACC: Basic to Translational Science Interview With ChatGPT. JACC. Basic Transl. Sci. 2023;8:221–223. doi: 10.1016/j.jacbts.2023.01.001. - DOI - PMC - PubMed
    1. Graber-Stiehl I. Is the world ready for ChatGPT therapists? Nature. 2023;617:22–24. doi: 10.1038/d41586-023-01473-4. - DOI - PubMed
    1. Khan R.A., Jawaid M., Khan A.R., Sajjad M. ChatGPT-Reshaping medical education and clinical management. Pakistan J. Med. Sci. 2023;39:605–607. - PMC - PubMed
    1. Galido P.V., Butala S., Chakerian M., Agustines D. A Case Study Demonstrating Applications of ChatGPT in the Clinical Management of Treatment-Resistant Schizophrenia. Cureus. 2023;15:e38166. - PMC - PubMed

LinkOut - more resources