Deep Learning for Diagnosing and Segmenting Choroidal Neovascularization in OCT Angiography in a Large Real-World Data Set

Abstract

Purpose: To diagnose and segment choroidal neovascularization (CNV) in a real-world multicenter clinical OCT angiography (OCTA) data set using deep learning.

Methods: A total of 105,66 OCTA scans from 3135 eyes, including 4701 with CNV and 5865 without, were collected in five eye clinics. Both 3 × 3-mm and 6 × 6-mm scans of the central and temporal macula were included. Scans with CNV were collected from multiple diseases, and scans without CNV were collected from both healthy controls and those with multiple diseases. No scans were removed during training or testing due to poor quality. The trained hybrid multitask convolutional neural network outputs a CNV diagnosis and membrane segmentation, respectively.

Results: The model demonstrated a highly accurate CNV diagnosis (area under receiver operating characteristic curve = 0.97), achieving a sensitivity of 95% at 95% specificity. The model also correctly segmented CNV lesions (F1 score = 0.78 ± 0.19). Additionally, model performance was comparable on both high-definition 3 × 3-mm scans and low-definition 6 × 6-mm scans. The model did not suffer large performance variations under different diseases. We also show that a subclinical lesion in a patient with neovascular age-related macular degeneration can be monitored over a multiyear time frame using our approach.

Conclusions: The proposed method can accurately diagnose and segment CNV in a large real-world clinical data set.

Translational relevance: The algorithm could enable automated CNV screening and quantification in the clinic, which will help improve CNV diagnosis and treatment evaluation.

Figure 1.

Proposed CNV diagnosis and segmentation method. (A) OCTA data was preprocessed by removing projection artifacts and segmenting retinal layers; OCTA scans were further graded as CNV/non-CNV scans, and CNV membrane area was outlined in a projection-resolved outer retinal angiogram. (B) Model inputs consist of an en face OCTA image set, thickness maps, an en face structural OCT image set, and outer retinal OCT and OCTA volumes. The OCTA image set consists of uncorrected inner and outer retinal angiograms, a slab-subtracted outer retinal angiogram, and a projection-resolved outer retinal angiogram, all produced using maximum projection. The structural OCT image set includes en face mean projections of 10 equal partitions of the entire data volume as well as a mean projection covering just the inner retina and a mean projection of the entire volume. (C) The proposed hybrid multitask CNN model combines 3D and 2D convolution; the feature extraction block is shared between the CNV diagnosis and membrane segmentation modules. F1 to F8 are tensor connection tags.

Figure 2.

A representative case in which follow-ups with large, apparent changes were considered independent samples.

Figure 3.

Representative CNV membrane segmentation results. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) Output CNV membrane probability map. (D) Structural OCT cross section at the location of the white line in A, with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Row 1: example in which the network achieved an accurate segmentation. Row 2: example of undersegmentation. The lesion in this scan extended beyond the OCTA scan's field of view, which may be difficult for the network to analyze properly due to the truncated nature of the information available at the edge of the image. Row 3: an example of oversegmentation and low prediction confidence in the CNV membrane area. Note that the pathologic flow signal in the outer retina was of similar magnitude to the background in this scan, making analysis difficult. Row 4: an example of a segmentation error in an extremely low-quality scan. This eye exhibits extreme defocus and a large number of residual artifacts. Note that the network did make a low-confidence prediction of a CNV membrane in a region that overlapped with the ground truth.

Figure. 4.

CNV membrane segmentation on both 3 × 3-mm and 6 × 6-mm scans. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) CNV probability map. (D) Structural OCT cross section at the location of the white line in (A), with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Case 1 shows a large CNV lesion that cannot be fully imaged using a small scanning window. Case 2 shows a small CNV lesion in a defocused scan with strong residual artifacts. Both cases at both resolutions resulted in correct detections.

Figure 5.

CNV membrane segmentation results in several retinal diseases. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) CNV probability map. (D) Structural OCT cross section at the location of the white line in (A), with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Row 1: example segmentation in neovascular AMD. Row 2: example segmentation in angiomatous proliferation. The identity of the lesion is best appreciated in the cross-sectional image (D2). Despite the small lesion size, this scan resulted in a correct prediction. Row 3: example segmentation in polypoidal choroidal vasculopathy (PCV). The flow signal in PCV is often very weak, which makes it difficult to distinguish from the background (B3). Row 4: example segmentation in pathologic myopia. Eyes with pathologic myopia often have extreme defocus in OCTA imaging (A4, B4). The residual artifacts in these scans probably account for the relatively low performance (C4).

Figure 6.

CNV growth dynamics illuminated by the proposed detection algorithm. CNV vessels were detected in the segmented CNV membrane area (white outline) using Otsu's algorithm. CNV vascular patterns changed dramatically after treatment. The scatterplot of the CNV vessel area and membrane area shows the CNV dynamics following anti-vascular endothelial growth factor (VEGF) treatments (red arrows) in longitudinal follow-ups.