Preliminary analysis of predicting the first recurrence in patients with neovascular age-related macular degeneration using deep learning

Abstract

Background: To predict, using deep learning, the first recurrence in patients with neovascular age-related macular degeneration (nAMD) after three monthly loading injections of intravitreal anti-vascular endothelial growth factor (anti-VEGF).

Methods: Optical coherence tomography (OCT) images were obtained at baseline and after the loading phase. The first recurrence was defined as the initial appearance of a new retinal hemorrhage or intra/subretinal fluid accumulation after the initial resolution of exudative changes after three loading injections. Standard U-Net architecture was used to identify the three retinal fluid compartments, which include pigment epithelial detachment, subretinal fluid, and intraretinal fluid. To predict the first recurrence of nAMD, classification learning was conducted to determine whether the first recurrence occurred within three months after the loading phase. The recurrence classification architecture was built using ResNet50. The model with retinal regions of interest of the entire region and fluid region on OCT at baseline and after the loading phase is presented.

Results: A total of 1,444 eyes of 1,302 patients were included. The mean duration until the first recurrence after the loading phase was 8.20 ± 15.56 months. The recurrence classification system revealed that the model with the fluid region of OCT after the loading phase provided the highest classification performance, with an area under the receiver operating characteristic curve (AUC) of 0.725 ± 0.012. Heatmap analysis revealed that three pathological fluids, subsided choroidal neovascularization lesions, and hyperreflective foci were important areas for the first recurrence.

Conclusions: The deep learning algorithm allowed for the prediction of the first recurrence for three months after the loading phase with adequate feasibility. An automated prediction system may assist in establishing patient-specific treatment plans and the provision of individualized medical care for patients with nAMD.

Keywords: Anti-VEGF; Deep learning; Neovascular age-related macular degeneration; Optical coherence tomography; Recurrence prediction.

Fig. 1

A schematic of deep learning model training, validation, and performance assessment for predicting the first recurrence of neovascular age-related macular degeneration patients using optical coherence tomography (OCT) image sets. A OCT scans were obtained at baseline and after the loading phase, which was taken one month after three consecutive anti-vascular endothelial growth factor (anti-VEGF) loading injections. B Retinal regions of interest (ROIs) were found using a fluid segmentation network. Recurrence classification network (ResNet50) (C) and gradient-weighted class activation mapping visualization (D) is presented

Fig. 2

Convolutional neural network-based fluid segmentation results and segmentation color key. A Representative original images of optical coherence tomography (OCT) scans. B OCT scans with retinal fluid regions. All retinal fluid was coded in yellow. C OCT scans with three different retinal fluids. Pigment epithelial detachment (PED) was coded in pink, subretinal fluid (SRF) in green, and intraretinal fluid (IRF) in blue

Fig. 3

Receiver operating characteristic (ROC) curves for convolutional neural network-based recurrence classification. ROC curves for the entire region (A) and the fluid region (B) of optical coherence tomography (OCT) scans at baseline. ROC curves for the entire region (C) and the fluid region (D) of OCT scans after the loading phase

Fig. 4

Representative cases of gradient-weighted class activation mapping (Grad-CAM) visualization. Grad-CAM extracts the feature map of the last convolutional layer and shows a heatmap within the image describing the calculated weight of the feature map. A True positive; B False positive; C False negative; D True negative cases

Fig. 5

Representative cases of neovascular age-related macular degeneration. Optical coherence tomography (OCT) scans at baseline, after the loading phase, and at the time of the first recurrence are presented. Convolutional neural network-based fluid segmentation results and gradient-weighted class activation mapping (Grad-CAM) are also noted. The red bounding box indicates the area shown by the Grad-CAM. A OCT demonstrates the first recurrence at 1.1499 months (within three months) after the loading phase, predicting a value higher than 0.5 with a prediction of 0.6391, which is true positive. B OCT demonstrates the first recurrence at 7.3595 months (after three months) after the loading phase, predicting a value lower than 0.5 with a prediction of 0.1362, which is true negative