Development of a deep learning model to distinguish the cause of optic disc atrophy using retinal fundus photography

Abstract

The differential diagnosis for optic atrophy can be challenging and requires expensive, time-consuming ancillary testing to determine the cause. While Leber's hereditary optic neuropathy (LHON) and optic neuritis (ON) are both clinically significant causes for optic atrophy, both relatively rare in the general population, contributing to limitations in obtaining large imaging datasets. This study therefore aims to develop a deep learning (DL) model based on small datasets that could distinguish the cause of optic disc atrophy using only fundus photography. We retrospectively reviewed fundus photographs of 120 normal eyes, 30 eyes (15 patients) with genetically-confirmed LHON, and 30 eyes (26 patients) with ON. Images were split into a training dataset and a test dataset and used for model training with ResNet-18. To visualize the critical regions in retinal photographs that are highly associated with disease prediction, Gradient-Weighted Class Activation Map (Grad-CAM) was used to generate image-level attention heat maps and to enhance the interpretability of the DL system. In the 3-class classification of normal, LHON, and ON, the area under the receiver operating characteristic curve (AUROC) was 1.0 for normal, 0.988 for LHON, and 0.990 for ON, clearly differentiating each class from the others with an overall total accuracy of 0.93. Specifically, when distinguishing between normal and disease cases, the precision, recall, and F1 scores were perfect at 1.0. Furthermore, in the differentiation of LHON from other conditions, ON from others, and between LHON and ON, we consistently observed precision, recall, and F1 scores of 0.8. The model performance was maintained until only 10% of the pixel values of the image, identified as important by Grad-CAM, were preserved and the rest were masked, followed by retraining and evaluation.

Keywords: Deep learning; Fundus photography; Leber hereditary optic neuropathy; Optic neuritis; Optic neuropathy.

Figure 1

Overview of the proposed study flow.

Figure 2

Receiver operating characteristic curves. The yellow curve represents how the model distinguishes normal from other classes, the green curve represents how the model distinguishes LHON from other classes, the red curve represents how the model distinguishes ON from other classes. The AUROC is represented in each legend. LHON, Leber’s hereditary optic neuropathy; ON, optic neuritis; AUROC, area under the receiver operating characteristic curve.

Figure 3

Heatmap of accuracy, precision, recall, and F1 score of model between classes. LHON, Leber’s hereditary optic neuropathy; ON, optic neuritis. A comparison between LHON and “Rest” is a comparison between LHON and both ON and normal.

Figure 4

Representative retinal fundus photographs. In each row, the first image shows the images after cropping, the second image shows the input of the model, the third image shows which areas the model paid attention to, and the fourth image is the heatmap of Grad-CAM and shows the intensity of attention. Grad-CAM, Gradient-Weighted Class Activation Map; LHON, Leber’s hereditary optic neuropathy; ON, optic neuritis.

Figure 5

Progressive erasure for each model, and AUC scores from the test set under the blurred images whose attention heatmap values are under the specific quantile. AUROC, area under the receiver operating characteristic curve; LHON, Leber’s hereditary optic neuropathy; ON, optic neuritis.