A deep learning system for detecting diabetic retinopathy across the disease spectrum

Abstract

Retinal screening contributes to early detection of diabetic retinopathy and timely treatment. To facilitate the screening process, we develop a deep learning system, named DeepDR, that can detect early-to-late stages of diabetic retinopathy. DeepDR is trained for real-time image quality assessment, lesion detection and grading using 466,247 fundus images from 121,342 patients with diabetes. Evaluation is performed on a local dataset with 200,136 fundus images from 52,004 patients and three external datasets with a total of 209,322 images. The area under the receiver operating characteristic curves for detecting microaneurysms, cotton-wool spots, hard exudates and hemorrhages are 0.901, 0.941, 0.954 and 0.967, respectively. The grading of diabetic retinopathy as mild, moderate, severe and proliferative achieves area under the curves of 0.943, 0.955, 0.960 and 0.972, respectively. In external validations, the area under the curves for grading range from 0.916 to 0.970, which further supports the system is efficient for diabetic retinopathy grading.

Fig. 1. Data split in the local dataset (SIM cohort) for the training and local validation of the three sub-networks of the DeepDR system.

The local dataset was randomly divided into training or validation datasets. All 466,247 images in the training dataset were used for training the image quality assessment sub-network. The lesion detection sub-network was trained using 10,280 gradable images with retinal lesion annotations. Then, 415,139 gradable images in the training set were used for the training of the DR grading sub-network. All 200,136 images in the local validation dataset were used to test the image quality sub-network, and 178,907 gradable images were used to test the DR grading sub-network. Finally, 4621 gradable images labeled with retinal lesions were used to test the lesion detection sub-network. DR, diabetic retinopathy.

Fig. 2. Visual diagram of the DeepDR system.

DeepDR system consisted of three sub-networks: image quality assessment sub-network, lesion-aware sub-network, and DR grading sub-network. We first pre-trained the ResNet to form the DR base network (top row). The trained weights of the pre-trained DR base network were then shared in the three different sub-networks of the system, indicated by the red arrow. These three sub-networks took retinal images as input and performed different tasks one-by-one. Furthermore, the lesion features extracted by the segmentation module of the lesion-aware sub-network (indicated by the green arrow) were concatenated with the features extracted by the DR grading sub-network (indicated by the blue arrow). DR, diabetic retinopathy; NPDR, non-proliferative diabetic retinopathy; PDR, proliferative diabetic retinopathy.

Fig. 3. Performance of the lesion-aware sub-network.

A Receiver operating characteristic curve demonstrating the performance of the lesion-aware sub-network for retinal lesion detection (n = 4621). B Example images of retinal lesion segmentation: microaneurysms, cotton-wool spots, hard exudates, and hemorrhages are highlighted using green regions.