Automated Segmentation of Retinal Fluid Volumes From Structural and Angiographic Optical Coherence Tomography Using Deep Learning

Abstract

Purpose: We proposed a deep convolutional neural network (CNN), named Retinal Fluid Segmentation Network (ReF-Net), to segment retinal fluid in diabetic macular edema (DME) in optical coherence tomography (OCT) volumes.

Methods: The 3- × 3-mm OCT scans were acquired on one eye by a 70-kHz OCT commercial AngioVue system (RTVue-XR; Optovue, Inc., Fremont, CA, USA) from 51 participants in a clinical diabetic retinopathy (DR) study (45 with retinal edema and six healthy controls, age 61.3 ± 10.1 (mean ± SD), 33% female, and all DR cases were diagnosed as severe NPDR or PDR). A CNN with U-Net-like architecture was constructed to detect and segment the retinal fluid. Cross-sectional OCT and angiography (OCTA) scans were used for training and testing ReF-Net. The effect of including OCTA data for retinal fluid segmentation was investigated in this study. Volumetric retinal fluid can be constructed using the output of ReF-Net. Area-under-receiver-operating-characteristic-curve, intersection-over-union (IoU), and F1-score were calculated to evaluate the performance of ReF-Net.

Results: ReF-Net shows high accuracy (F1 = 0.864 ± 0.084) in retinal fluid segmentation. The performance can be further improved (F1 = 0.892 ± 0.038) by including information from both OCTA and structural OCT. ReF-Net also shows strong robustness to shadow artifacts. Volumetric retinal fluid can provide more comprehensive information than the two-dimensional (2D) area, whether cross-sectional or en face projections.

Conclusions: A deep-learning-based method can accurately segment retinal fluid volumetrically on OCT/OCTA scans with strong robustness to shadow artifacts. OCTA data can improve retinal fluid segmentation. Volumetric representations of retinal fluid are superior to 2D projections.

Translational relevance: Using a deep learning method to segment retinal fluid volumetrically has the potential to improve the diagnostic accuracy of diabetic macular edema by OCT systems.

Keywords: OCT/OCTA; deep learning; retinal fluid volume; segmentation.

Figure 1.

The architecture of deep convolutional neural network constructed in this study. (A) ReF-Net architecture. (B) Multi-scale block. (C, D) Residual convolutional blocks.

Figure 2.

Representative OCT/OCTA B-scan showing retinal fluid. (A) OCT B-scan. (B) OCTA B-scan. (C) Ground truth map with three categories, background (green), retinal tissue (black), and retinal fluid area (red).

Figure 3.

Manual delineation of the ground truth for training. (A) The in-house graphical user interface software. (B) Three graders manually delineated the background (green), retinal tissue (black), and retinal fluid area (red). Pixel-wise voting method to generate the final ground truth map.

Figure 4.

Comparison between ReF-Net-OCTA (β = 0.20) and ground truth on structural OCT B-scans. (A) Structural OCT B-scans. (B) Segmented fluid maps from ReF-Net (blue) and (C) the ground truth maps (red) overlaid on structural cross-sections. (D) Difference map between segmented fluid from ReF-Net and ground truth. White area is the overlap region of two maps. The blue and red in (D) show pixels exclusively in the algorithm output or ground truth, respectively.

Figure 5.

Automated retinal fluid segmentation results on shadow artifact effected scans. Yellow arrows indicate shadow artifacts. (Row A) Example case with large vessel shadow artifacts. (Row B) Example case with vitreous floater shadow artifacts. (Row C) Example case with pupil vignetting shadow artifacts. (Column 1) Reflectance en face images, with the green line indicating the position of the B-scan shown in the other columns. (Column 2) Raw cross-sectional scans. (Column 3) Ground truth map (red) overlaid on B-scans. (Column 4) ReF-Net (ReF-Net-OCTA, β = 0.20) outputs (blue) overlaid on the B-scans.

Figure 6.

Comparison between 2D projected fluid areas and 3D fluid volumes in DME cases. (A1-D1) 2D structural OCT and retinal fluid projections. (A2-D2) 3D structural OCT and retinal fluid representations. Apparent fluid areas can be similar while volumes are quite different (A, B), and apparent fluid areas can be quite different while volumes are similar (C, D). In such cases, the 2D projection is misleading.

Figure 7.

A DME case in which a substantial portion of retinal fluid would be missed by under-sampled scans. (A) Infrared photograph with sampling positions (green lines) from a Spectralis OCT (Heidelberg Engineering Inc.) scan. (B) Dense volumetric OCT (RTVue-XR; Optovue, Inc.) with retinal fluid volume (blue). The yellow square in (A) indicates the scanning position in (B). Red arrows indicate the retinal fluid missed by the undersampled scan, which can be detected by our algorithm using the densely-sampled OCT. Green lines indicate the sampling position of Spectralis OCT scan.

Figure 8.

A diabetic macular edema (DME) case with a false-negative result from central macular thickness (CMT) was automatically detected and measured by ReF-Net. (A) Retinal fluid volume segmented by ReF-Net. (B) Cross-sectional structural OCT. (C) Retinal thickness map and average thickness distribution in early treatment diabetic retinopathy study (ETDRS) grid. The CMT value is 217, which does not meet the definition of DME.

Figure 9.

Local dynamics of retinal fluid in longitudinal monitoring of a DME eye. (A) Baseline. (B) One year follow-up after the treatment. (C) Registered baseline and follow-up scans. (D) Changes in the retinal fluid region. (E) Baseline retinal fluid area overlaid on an inner retinal OCT angiogram. (F) Follow-up retinal fluid area overlaid on an inner retinal OCT angiogram. The yellow arrow indicated the change of vasculature caused by retinal fluid.