Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
[Preprint]. 2025 Jul 7:2025.07.03.25330805.
doi: 10.1101/2025.07.03.25330805.

GlaucoRAG: A Retrieval-Augmented Large Language Model for Expert-Level Glaucoma Assessment

Mohammad Aminan  1 , S Solomon Darnell  2 , Mohammad Delsoz  1 , Amin Nabavi  1 , Claire Wright  1 , Brian Jerkins  1 , Siamak Yousefi  1
Affiliations

GlaucoRAG: A Retrieval-Augmented Large Language Model for Expert-Level Glaucoma Assessment

Mohammad Aminan et al. medRxiv. .

Abstract

Purpose: Purpose: Accurate glaucoma assessment is challenging because of the complexity and chronic nature of the disease; therefore, there is a critical need for models that provide evidence-based, accurate assessment. The purpose of this study was to evaluate the capabilities of a glaucoma specialized Retrieval-Augmented Generation (RAG) framework (GlaucoRAG) that leverages a large language model (LLM) for diagnosing glaucoma and answering to glaucoma specific questions.

Design: Evaluation of diagnostic capabilities and knowledge of emerging technologies in glaucoma assessment.

Participants: Detailed case reports from 11 patients and 250 multiple choice questions from the Basic and Clinical Science Course (BCSC) Self-Assessment were used to test the LLM based GlaucoRAG. No human participants were involved.

Methods: We developed GlaucoRAG, a RAG framework leveraging GPT-4.5-PREVIEW integrated with the R2R platform for automated question answering in glaucoma. We created a glaucoma knowledge base comprising more than 1,800 peer-reviewed glaucoma articles, 15 guidelines and three glaucoma textbooks. The diagnostic performance was tested on case reports and multiple-choice questions. Model outputs were compared with the independent answers of three glaucoma specialists, DeepSeek-R1, and GPT-4.5-PREVIEW (without RAG). Quantitative performance was further assessed with the RAG Assessment (RAGAS) framework, reporting faithfulness, context precision, context recall, and answer relevancy.

Main outcome measures: The primary outcome measure was GlaucoRAG's diagnostic accuracy on patient case reports and percentage of correct responses to the BCSC Self-Assessment glaucoma items, compared with the performance of glaucoma specialists and two benchmark LLMs. Secondary outcomes included RAGAS sub scores.

Results: GlaucoRAG achieved an accuracy of 81.8% on glaucoma case reports, compared with 72.7% for GPT-4.5-PREVIEW and 63.7% for DeepSeek-R1. On glaucoma BCSC Self-Assessment questions, GlaucoRAG achieved 91.2% accuracy (228 / 250), whereas GPT-4.5-PREVIEW and DeepSeek-R1 attained 84.4% (211 / 250) and 76.0% (190 / 250), respectively. The RAGAS evaluation returned an answer relevancy of 91%, with 80% context recall, 70% faithfulness, and 59% context precision.

Conclusions: The glaucoma-specialized LLM, GlaucoRAG, showed encouraging performance in glaucoma assessment and may complement glaucoma research and clinical practice as well as question answering with glaucoma patients.

Keywords: Glaucoma; Glaucoma Specialized RAG (GlaucoRAG); Large Language Mdoel (LLM); Question Answering (QA); Retrieval-Augmented Generation (RAG).

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Architecture of the glaucoma specialized retrieval augmented generation (GlaucoRAG) framework.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Madadi Yeganeh, Delsoz Mohammad, Khouri Albert S., Boland Michael, Grzybowski Andrzej, and Yousefi Siamak. Applications of artificial intelligence-enabled robots and chatbots in ophthalmology: recent advances and future trends. Current Opinion in Ophthalmology, 35(3), 2024. - PMC - PubMed

    2. N2 - Purpose of review Recent advances in artificial intelligence (AI), robotics, and chatbots have brought these technologies to the forefront of medicine, particularly ophthalmology. These technologies have been applied in diagnosis, prognosis, surgical operations, and patient-specific care in ophthalmology. It is thus both timely and pertinent to assess the existing landscape, recent advances, and trajectory of trends of AI, AI-enabled robots, and chatbots in ophthalmology. Recent findings Some recent developments have integrated AI enabled robotics with diagnosis, and surgical procedures in ophthalmology. More recently, large language models (LLMs) like ChatGPT have shown promise in augmenting research capabilities and diagnosing ophthalmic diseases. These developments may portend a new era of doctor-patient-machine collaboration. Summary Ophthalmology is undergoing a revolutionary change in research, clinical practice, and surgical interventions. Ophthalmic AI-enabled robotics and chatbot technologies based on LLMs are converging to create a new era of digital ophthalmology. Collectively, these developments portend a future in which conventional ophthalmic knowledge will be seamlessly integrated with AI to improve the patient experience and enhance therapeutic outcomes.

    1. Madadi Yeganeh, Abu-Serhan Hashem, and Yousefi Siamak. Domain Adaptation-Based deep learning model for forecasting and diagnosis of glaucoma disease. Biomedical Signal Processing and Control, 92:106061, June 2024. - PMC - PubMed
    1. Yousefi Siamak, Pasquale Louis R., Boland Michael V., and Johnson Chris A.. Machine-Identified Patterns of Visual Field Loss and an Association with Rapid Progression in the Ocular Hypertension Treatment Study. Ophthalmology, 129(12):1402–1411, December 2022. - PMC - PubMed
    1. Yousefi Siamak, Elze Tobias, Pasquale Louis R., Saeedi Osamah, Wang Mengyu, Shen Lucy Q., Wellik Sarah R., De Moraes Carlos G., Myers Jonathan S., and Boland Michael V.. Monitoring Glaucomatous Functional Loss Using an Artificial Intelligence–Enabled Dashboard. Ophthalmology, 127(9):1170–1178, September 2020. - PMC - PubMed
    1. Thakur Anshul, Goldbaum Michael, and Yousefi Siamak. Predicting Glaucoma before Onset Using Deep Learning. Ophthalmology Glaucoma, 3(4):262–268, July 2020. - PMC - PubMed

Publication types

LinkOut - more resources