Development of a Deep-Learning-Based Artificial Intelligence Tool for Differential Diagnosis between Dry and Neovascular Age-Related Macular Degeneration

Abstract

The use of deep-learning-based artificial intelligence (AI) is emerging in ophthalmology, with AI-mediated differential diagnosis of neovascular age-related macular degeneration (AMD) and dry AMD a promising methodology for precise treatment strategies and prognosis. Here, we developed deep learning algorithms and predicted diseases using 399 images of fundus. Based on feature extraction and classification with fully connected layers, we applied the Visual Geometry Group with 16 layers (VGG16) model of convolutional neural networks to classify new images. Image-data augmentation in our model was performed using Keras ImageDataGenerator, and the leave-one-out procedure was used for model cross-validation. The prediction and validation results obtained using the AI AMD diagnosis model showed relevant performance and suitability as well as better diagnostic accuracy than manual review by first-year residents. These results suggest the efficacy of this tool for early differential diagnosis of AMD in situations involving shortages of ophthalmology specialists and other medical devices.

Keywords: age-related macular degeneration; class activation map; convolutional neural network; cross-validation; retina.

Figure 1

The proposed convolutional neural network (CNN) architecture (a modified Visual Geometry Group with 16 layers (VGG16) model). The CNN with the modified VGG16 model used 3 × 3 convolutional layers and 2 × 2 pooling layers. Convolutional layers and fully connected layers were trained with macular images.

Figure 2

Multimodal images of neovascular age-related macular degeneration in a 61-year-old man. (a) Fundus photography shows subretinal fluid, exudation, and hemorrhage; (b) Optical coherence tomography (OCT) B-scan revealed non-uniform hyper-reflective formations above the retinal pigment epithelium and the presence of intraretinal cysts and subretinal fluid; (c) Fluorescein angiography (FA) demonstrates aspects of a well-defined (white arrow) and an irregular (yellow arrow) hyper-fluorescent lesion; (d) Indocyanine green angiography (ICGA) shows staining of the type 2 choroidal neovascularization (CNV) (white arrow); (e) An OCT angiography image (with the neovascular network) overlaid on the ICGA image.

Figure 3

Fundus photography and optical coherence tomography of dry age-related macular degeneration (dAMD) and control retinas. (a) Numerous soft, yellow drusen in the right eye of a 78-year-old woman; (b) The corresponding OCT image shows multiple deposits accumulating under the retinal pigment epithelium. (c) Normal control fundus photography in the right eye of a 66-year-old man. (d) The corresponding OCT image of the control.

Figure 4

Image preprocessing. Eye images were preprocessed by Keras ImageDataGenerator. Original images were cropped and resized to 244 × 244 pixels. The training dataset images were generated using various methods, including width shift, height shift, rotation, zoom, horizontal flip, and vertical flip.

Figure 5

Examples of class activation map (CAM) visualization. CAM visualization of normal, dry age-related macular degeneration (dAMD), and neovascular age-related macular degeneration (nAMD) retinas. CAM extracts the feature map of the last convolution layer (Conv5_3) and shows a heatmap within the image describing the calculated weight of the feature map. (a) dAMD fundus images show drusen (arrow), and (d) heatmap images show drusen identified by the artificial intelligence (AI) tool; (b) Normal fundus images have no drusen, and (e) heatmap images of normal controls show that the AI tool identified the contour of fovea according to the absence of drusen; (c) nAMD fundus images show bleeding and degenerated areas (green arrows), and (f) heatmap images show identified drusen and other features of degeneration and bleeding; (g–i) Representative images of dAMD, a normal control, and nAMD, respectively; (l) Heatmap images of nAMD show that the AI tool identified pathological changes in the macula, such as elevation of the center; (j) There was no heatmap at the center of dAMD; however, the AI tool detected drusen instead; (k) Heatmap image showing AI identification of the center of the macula in a control, with no degenerated area.