Examining the competence of artificial intelligence programs in neuro-ophthalmological disorders and analyzing their comparative superiority

Abstract

Background: This study aims to evaluate the knowledge levels of chat generative pretrained transformer (ChatGPT), Bing, and Bard programs, which are three different artificial intelligence chatbots offered to the market free of charge by various manufacturers, regarding neuro-ophthalmological diseases, to examine their usability, and to investigate the existence of their superiority to each other.

Materials and methods: Forty questions related to neuro-ophthalmological diseases were obtained from the study questions' section of the American Academy and Ophthalmology 2022-2023 Basic and Clinical Science Course Neuro-ophthalmology Book. The questions were posed to the ChatGPT, Bing, and Bard artificial intelligence chatbots. The answers were evaluated as correct or incorrect. The statistical relationship between the correct and incorrect answer rates offered by the artificial intelligence programs was tested.

Results: The correct answer rates were given by the artificial intelligence programs to the questions asked: ChatGPT - 52.5%; Bing - 55%; and Bard - 65%. There was no statistically significant difference between the correct answer rates of the three artificial intelligence programs (P = 0.489, Pearson's Chi-square test).

Conclusion: Although information about neuro-ophthalmological diseases can be accessed quickly and accurately using up-to-date artificial intelligence programs, the answers given may not always be correct. Care should always be taken when evaluating the answers to the questions.

Keywords: Artificial intelligence; Bard; Bing; chat generative pretrained transformer; neuro-ophthalmology.