Explainable end-to-end deep learning for diabetic retinopathy detection across multiple datasets

Abstract

Purpose: Diabetic retinopathy (DR) is characterized by retinal lesions affecting people having diabetes for several years. It is one of the leading causes of visual impairment worldwide. To diagnose this disease, ophthalmologists need to manually analyze retinal fundus images. Computer-aided diagnosis systems can help alleviate this burden by automatically detecting DR on retinal images, thus saving physicians' precious time and reducing costs. The objective of this study is to develop a deep learning algorithm capable of detecting DR on retinal fundus images. Nine public datasets and more than 90,000 images are used to assess the efficiency of the proposed technique. In addition, an explainability algorithm is developed to visually show the DR signs detected by the deep model. Approach: The proposed deep learning algorithm fine-tunes a pretrained deep convolutional neural network for DR detection. The model is trained on a subset of EyePACS dataset using a cosine annealing strategy for decaying the learning rate with warm up, thus improving the training accuracy. Tests are conducted on the nine datasets. An explainability algorithm based on gradient-weighted class activation mapping is developed to visually show the signs selected by the model to classify the retina images as DR. Result: The proposed network leads to higher classification rates with an area under curve (AUC) of 0.986, sensitivity = 0.958, and specificity = 0.971 for EyePACS. For MESSIDOR, MESSIDOR-2, DIARETDB0, DIARETDB1, STARE, IDRID, E-ophtha, and UoA-DR, the AUC is 0.963, 0.979, 0.986, 0.988, 0.964, 0.957, 0.984, and 0.990, respectively. Conclusions: The obtained results achieve state-of-the-art performance and outperform past published works relying on training using only publicly available datasets. The proposed approach can robustly classify fundus images and detect DR. An explainability model was developed and showed that our model was able to efficiently identify different signs of DR and detect this health issue.

Keywords: convolutional neural networks; diabetic retinopathy; exudates and hemorrhage; inception; microaneurysms; residual networks.

Fig. 1

Examples of (a) a normal retina and (b) a DR damaged retina.

Fig. 2

Different signs of DR on fundus images: (a) EX, (b) HM, and (c) MA.

Fig. 3

Proposed approach for DR detection.

Fig. 4

Inception-Resnet-v2 blocks: (a) Inception-Resnet-A, (b) Inception-Resnet-B, and (c) Inception-Resnet-C.

Fig. 5

The customized network with Stem module.

Fig. 6

Sample fundus images in MESSIDOR and MESSIDOR-2 datasets.

Fig. 7

Sample fundus images in the E-ophtha dataset.

Fig. 8

Sample fundus images in DIARETDB0 and DIARETDB1 datasets.

Fig. 9

Sample fundus images in the STARE dataset.

Fig. 10

Sample fundus images in the IDRID dataset.

Fig. 11

Sample fundus images in the UoA-DR dataset.

Fig. 12

Sample fundus images in the EyePACS dataset.

Fig. 13

EyePACS dataset distribution.

Fig. 14

Learning curves for training with the nine configurations: (a) ACC and (b) loss.

Fig. 15

Learning curves for validation with the nine configurations: (a) ACC and (b) loss.

Fig. 16

ROC curves for the nine configurations.

Fig. 17

ROC curves for the nine datasets.

Fig. 18

Examples of true positive and true negative on EyePACS: (a) TP for EX and HM detected near the optic disc, (b) TN with no sign of DR, (c) TP includes EX, HM, and MA located on the right of the retina, and (d) TP for detected HM.

Fig. 19

Examples of true positive and true negative on MESSIDOR and MESSIDOR-2 datasets: (a) TP for EX near macula, (b) TP includes detection of MA, (c) TP for EX detection, and (d) TN with no sign of DR.

Fig. 20

Examples of true positive and true negative on DIARETDB0 and DIARETDB1 datasets: (a), (b) TP includes detection of MA and HM, (c) TN with no sign of DR, and (d) TP includes detection for soft EX, MA, and HM.

Fig. 21

Examples of true positive and true negative on STARE dataset: (a) TP including the detection of EX and HM, (b) TP includes detection of MA, (c) TN with no sign of DR, and (d) TP including the detection of HM.

Fig. 22

Examples of true positive and true negative on IDRID dataset: (a), (b) TP includes detection of EX and HM, (c) TP includes detection for hard EX and HM, and (c) TN with no sign of DR.

Fig. 23

Examples of true positive and true negative on E-ophtha dataset: (a), (b) TP includes detection of MA and hemorrhage, (c) TN with no sign of DR, and (d) TP includes detection for soft EX and MA.

Fig. 24

Examples of true positive and true negative on UoA-DR dataset: (a), (b) TP including the detection of EX and HM, (c) TN with no sign of DR, and (d) TP includes detection for EX and MA.

Fig. 25

Examples of ungradable images in the EyePACS dataset.

Fig. 26

FP images and FN images in nine datasets: (a), (d), (g), and (h) FP: EyePACS; (b), (c), (e), (f), and (i) FN: EyePACS; (j) FN: MESSIDOR-2; (k) FN: E-ophtha; (l) FN: DIARETDB0; (m) FP: DIARETDB1; (n) FP: IDRID; (o) FP: UoA-DR; and (p) FP: UoA-DR.