MSLF-Net: A Multi-Scale and Multi-Level Feature Fusion Net for Diabetic Retinopathy Segmentation

Abstract

Diabetic Retinopathy (DR) is a diabetic complication that predisposes patients to visual impairments that could lead to blindness. Lesion segmentation using deep learning algorithms is an effective measure to screen and prevent early DR. However, there are several types of DR with varying sizes and high inter-class similarity, making segmentation difficult. In this paper, we propose a supervised segmentation method (MSLF-Net) based on multi-scale-multi-level feature fusion to achieve accurate end-to-end DR lesion segmentation. MSLF-Net builds a Multi-Scale Feature Extraction (MSFE) module to extract multi-scale information and provide more comprehensive features for segmentation. This paper further introduces the Multi-Level Feature Fusion (MLFF) module to improve feature fusion using a cross-layer structure. This structure only fuses low- and high-level features of the same class based on category supervision, avoiding feature contamination. Moreover, this paper produces additional masked images for the dataset and performs image enhancement operations to ensure that the proposed method is trainable and functional on small datasets. The extensive experiments are conducted on public datasets IDRID and e_ophtha. The results showed that our proposed feature enhancement method can perform feature fusion more effectively. Therefore, In the end-to-end DR segmentation neural network model, MSLF Net is superior to other similar models in segmentation, and can effectively improve the DR lesion segmentation performance.

Keywords: diabetic retinopathy; feature fusion; image segmentation; multi-level; multi-scale.

Figure 1

Illustration of fundus image with DR lesion.

Figure 2

Illustration of fundus image, DR lesions, and FOV mask image.

Figure 3

The ovaerall architecture of MSLF-Net. (Green arrow denotes down-sampling, red arrow denotes up-sampling, black arrow denotes information flow, blue arrow denotes channel rearrange, orange arrow denotes channel fusion, n = number of lesion types + 1).

Figure 4

The stucture of the MSFE module.

Figure 5

Detailed description of simple feature fusion and multi-level feature fusion.

Figure 6

Learning rate curve and loss curve.

Figure 7

PR curves of MSLF-Net on IDRID.

Figure 8

PR curves of MSLF-Net on e_ophtha.

Figure 9

Illustration of fundus image with DR lesion. The rectangular box in the figure shows the comparison between the segmentation results of various methods and Ground Truth. The yellow boxes indicate that our method reduces the possibility of false detection compared with other methods, the green boxes indicate that our method reduces the possibility of false detection, while the red boxes indicate the error of our method.

Figure 10

Illustration of fundus image with DR lesion.