Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
[Preprint]. 2023 Aug 28:2023.08.25.23294635.
doi: 10.1101/2023.08.25.23294635.

Performance of ChatGPT in Diagnosis of Corneal Eye Diseases

Mohammad Delsoz  1 , Yeganeh Madadi  1 , Wuqaas M Munir  2 , Brendan Tamm  2 , Shiva Mehravaran  3 , Mohammad Soleimani  4   5 , Ali Djalilian  4 , Siamak Yousefi  1   6
Affiliations

Performance of ChatGPT in Diagnosis of Corneal Eye Diseases

Mohammad Delsoz et al. medRxiv. .

Update in

Abstract

Introduction: Assessing the capabilities of ChatGPT-4.0 and ChatGPT-3.5 for diagnosing corneal eye diseases based on case reports and compare with human experts.

Methods: We randomly selected 20 cases of corneal diseases including corneal infections, dystrophies, degenerations, and injuries from a publicly accessible online database from the University of Iowa. We then input the text of each case description into ChatGPT-4.0 and ChatGPT3.5 and asked for a provisional diagnosis. We finally evaluated the responses based on the correct diagnoses then compared with the diagnoses of three cornea specialists (Human experts) and evaluated interobserver agreements.

Results: The provisional diagnosis accuracy based on ChatGPT-4.0 was 85% (17 correct out of 20 cases) while the accuracy of ChatGPT-3.5 was 60% (12 correct cases out of 20). The accuracy of three cornea specialists were 100% (20 cases), 90% (18 cases), and 90% (18 cases), respectively. The interobserver agreement between ChatGPT-4.0 and ChatGPT-3.5 was 65% (13 cases) while the interobserver agreement between ChatGPT-4.0 and three cornea specialists were 85% (17 cases), 80% (16 cases), and 75% (15 cases), respectively. However, the interobserver agreement between ChatGPT-3.5 and each of three cornea specialists was 60% (12 cases).

Conclusions: The accuracy of ChatGPT-4.0 in diagnosing patients with various corneal conditions was markedly improved than ChatGPT-3.5 and promising for potential clinical integration.

Keywords: Artificial Intelligence (AI); ChatGPT; Corneal eye diseases; Generative Pre-trained Transformer (GPT); Large Language Models (LLM); Provisional Diagnosis.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Mohammad Delsoz: None. Yeganeh Madadi: None Wuqaas M Munir: None Brendan Tamm: None Shiva Mehravaran: None Mohammad Soleimani: None Ali Djalilian: None Siamak Yousefi: Remidio, M&S Technologies, Visrtucal Fields, InsihgtAEye, Enolink

Figures

Figure 1.
Figure 1.
A sample case description input into the ChatGPT-4.0 model and corresponding responses.
See this image and copyright information in PMC

References

    1. Yang AY, Chow J, Liu J. Corneal Innervation and Sensation: The Eye and Beyond. Yale J Biol Med. Mar 2018;91(1):13–21. - PMC - PubMed
    1. Clinic C. doi:https://my.clevelandclinic.org/health/diseases/8586-corneal-disease
    1. Solomon SD, Shoge RY, Ervin AM, et al. Improving Access to Eye Care: A Systematic Review of the Literature. Ophthalmology. Oct 2022;129(10):e114–e126. doi: 10.1016/j.ophtha.2022.07.012 - DOI - PubMed
    1. Gelston CD, Patnaik JL. Ophthalmology training and competency levels in care of patients with ophthalmic complaints in United States internal medicine, emergency medicine and family medicine residents. J Educ Eval Health Prof. 2019;16:25. doi: 10.3352/jeehp.2019.16.25 - DOI - PMC - PubMed
    1. Liu PR, Lu L, Zhang JY, Huo TT, Liu SX, Ye ZW. Application of Artificial Intelligence in Medicine: An Overview. Curr Med Sci. Dec 2021;41(6):1105–1115. doi: 10.1007/s11596-021-2474-3 - DOI - PMC - PubMed

Publication types