Development and evaluation of a deep learning system for screening real-world multiple abnormal findings based on ultra-widefield fundus images

Abstract

Purpose: To develop and evaluate a deep learning system for screening multiple abnormal findings including hemorrhages, drusen, hard exudates, cotton wool spots and retinal breaks using ultra-widefield fundus images.

Methods: The system consisted of three modules: (I) quality assessment module, (II) artifact removal module and (III) lesion recognition module. In Module III, a heatmap was generated to highlight the lesion area. A total of 4,521 UWF images were used for the training and internal validation of the DL system. The system was evaluated in two external validation datasets consisting of 344 images and 894 images from two other hospitals. The performance of the system in these two datasets was compared with or without Module II.

Results: In both external validation datasets, the deep learning system made better performance when recognizing lesions on processed images after Module II than on original images without Module II. Module II-enhanced preprocessing improved Module III's five-lesion recognition performance by an average of 6.73% and 14.4% areas under the curves, 14.47% and 19.62% accuracy in the two external validations.

Conclusion: Our system showed reliable performance for detecting MAF in real-world UWF images. For deep learning systems to recognize real-world images, the artifact removal module was indeed helpful.

Keywords: artifact removal; deep learning; multiple abnormal findings; real-world images; ultra-widefield fundus images.

FIGURE 1

The overview of our proposed deep learning-based screening system.

FIGURE 2

Confusion matrix for image quality assessment.

FIGURE 3

Validation loss with early stopping.

FIGURE 4

The receiver operating characteristic (ROC) of MAF on Zhenjiang Ruikang Hospital (ZRH) external validation dataset.

FIGURE 5

Areas under the curves (AUC) performance comparison of five methods.

FIGURE 6

Representative images of original/processed UWF images and corresponding heatmaps generated using gradient-weighted class activation mapping (Grad-CAM). (a) Images and heatmaps with drusen; (b) Images and heatmaps with hemorrhage; (c) Images and heatmaps with both hemorrhage and drusen.