A Practical Guide to Evaluating Artificial Intelligence Imaging Models in Scientific Literature

Abstract

Objective: Recent advances in artificial intelligence (AI) are revolutionizing ophthalmology by enhancing diagnostic accuracy, treatment planning, and patient management. However, a significant gap remains in practical guidance for ophthalmologists who lack AI expertise to effectively analyze these technologies and assess their readiness for integration into clinical practice. This paper aims to bridge this gap by demystifying AI model design and providing practical recommendations for evaluating AI imaging models in research publications.

Design: Educational review: synthesizing key considerations for evaluating AI papers in ophthalmology.

Participants: This paper draws on insights from an interdisciplinary team of ophthalmologists and AI experts with experience in developing and evaluating AI models for clinical applications.

Methods: A structured framework was developed based on expert discussions and a review of key methodological considerations in AI research.

Main outcome measures: A stepwise approach to evaluating AI models in ophthalmology, providing clinicians with practical strategies for assessing AI research.

Results: This guide offers broad recommendations applicable across ophthalmology and medicine.

Conclusions: As the landscape of health care continues to evolve, proactive engagement with AI will empower clinicians to lead the way in innovation while concurrently prioritizing patient safety and quality of care.

Financial disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Keywords: Artificial intelligence; Glaucoma detection; Machine learning; Ophthalmology.

Figure 1

Three variations of the same OCT image using data augmentation techniques. The original image (left) is modified to create new computer-generated versions by adding Gaussian noise (center) and applying a horizontal flip (right). These transformations simulate variability in the dataset, enhancing the AI model's ability to generalize. AI = artificial intelligence.

Figure 2

Grad-CAM highlights regions most important in the AI model's decision-making process, enhancing interpretability. In this case, the Grad-CAM (right) illustrates that model relies on the RNFL probability map's arcuate region to assess glaucoma, as indicated by the yellow highlights. This provides a visual explanation of the AI's focus during analysis. The color scale (yellow to blue) indicates the relevance of different regions, with yellow showing the highest relevance. AI = artificial intelligence; Grad-CAM = Gradient-Weighted Class Activation Maps; GCL = Ganglion Cell Layer; RNFL = retinal nerve fiber layer; VF = visual fields.

Figure 3

Precision-recall curve. AUC = area under the curve; PR = precision-recall.

Figure 4

Receiver operating characteristic curve. ROC = receiver operating characteristic.