Improving foveal avascular zone segmentation in fluorescein angiograms by leveraging manual vessel labels from public color fundus pictures

Abstract

In clinical routine, ophthalmologists frequently analyze the shape and size of the foveal avascular zone (FAZ) to detect and monitor retinal diseases. In order to extract those parameters, the contours of the FAZ need to be segmented, which is normally achieved by analyzing the retinal vasculature (RV) around the macula in fluorescein angiograms (FA). Computer-aided segmentation methods based on deep learning (DL) can automate this task. However, current approaches for segmenting the FAZ are often tailored to a specific dataset or require manual initialization. Furthermore, they do not take the variability and challenges of clinical FA into account, which are often of low quality and difficult to analyze. In this paper we propose a DL-based framework to automatically segment the FAZ in challenging FA scans from clinical routine. Our approach mimics the workflow of retinal experts by using additional RV labels as a guidance during training. Hence, our model is able to produce RV segmentations simultaneously. We minimize the annotation work by using a multi-modal approach that leverages already available public datasets of color fundus pictures (CFPs) and their respective manual RV labels. Our experimental evaluation on two datasets with FA from 1) clinical routine and 2) large multicenter clinical trials shows that the addition of weak RV labels as a guidance during training improves the FAZ segmentation significantly with respect to using only manual FAZ annotations.

Fig. 1.

FA scans showing the heterogeneity of clinical data. From top left to bottom right: A scan with (1) light motion blur and high contrast, (2) low contrast with almost no RV visible and black borders, (3) sufficient quality with the FAZ clearly visible, (4) a small FAZ and leaky RV, (5) non-centered fovea, (6) a scan with low contrast, (7) a huge FOV and (8) oversaturated brightness and contrast.

Fig. 2.

FA scans with their corresponding FAZ and RV segmentations obtained by the proposed approach.

Fig. 3.

Overview of the framework. 1.a) The CycleCAN learns the matching between CFP and FA 1.b) The CFP images are converted into pseudo-FA. 2.a) $F_{\text{RV-SEG}}$ is trained on the pseudo-FA and respective RV labels. 2.b.) $G_{\text{RV-SEG}}$ is applied on dataset $S_{\text{PUBLIC-CFP}}$ to obtain the weak labels. 3.a) The manual FAZ and weakly-supervised RV labels are combined into two-class labels. 3.b) $G_{\text{FAZ}\mathrm{\&}\text{RV-SEG}}$ is trained on dataset $S_{\text{REAL-FA}}$ with the two class labels.

Fig. 4.

Boxplots showing the Dice values for FAZ segmentation in testsets $S_{\text{REAL-FA}}$ and $S_{\text{EVAL}}$ from the standard U-Net model.

Fig. 5.

Qualitative examples of the results of our segmentation model in the testset from clinical trials. From left to right: FA input image, manually annotated FAZ and automated vessel segmentations used for training the model, FAZ segmentation produced by the baseline, FAZ and RV segmentation results produced by the proposed model.

Fig. 6.

Qualitative results obtained for converting CFP to pseudo-FA using our CycleGAN model. Left: input CFP. Right: pseudo-FA image converted from the CFP.

Fig. 7.

Qualitative results of RV segmentation on real FA images. From left to right: input image, manual annotations, prediction of our weakly trained model comparison.