ADS-Net: attention-awareness and deep supervision based network for automatic detection of retinopathy of prematurity

Abstract

Retinopathy of prematurity (ROP) is a proliferative vascular disease, which is one of the most dangerous and severe ocular complications in premature infants. Automatic ROP detection system can assist ophthalmologists in the diagnosis of ROP, which is safe, objective, and cost-effective. Unfortunately, due to the large local redundancy and the complex global dependencies in medical image processing, it is challenging to learn the discriminative representation from ROP-related fundus images. To bridge this gap, a novel attention-awareness and deep supervision based network (ADS-Net) is proposed to detect the existence of ROP (Normal or ROP) and 3-level ROP grading (Mild, Moderate, or Severe). First, to balance the problems of large local redundancy and complex global dependencies in images, we design a multi-semantic feature aggregation (MsFA) module based on self-attention mechanism to take full advantage of convolution and self-attention, generating attention-aware expressive features. Then, to solve the challenge of difficult training of deep model and further improve ROP detection performance, we propose an optimization strategy with deeply supervised loss. Finally, the proposed ADS-Net is evaluated on ROP screening and grading tasks with per-image and per-examination strategies, respectively. In terms of per-image classification pattern, the proposed ADS-Net achieves 0.9552 and 0.9037 for Kappa index in ROP screening and grading, respectively. Experimental results demonstrate that the proposed ADS-Net generally outperforms other state-of-the-art classification networks, showing the effectiveness of the proposed method.

Fig. 1.

Examples of normal and stage 1 to 5. The ROP lesion areas are in red boxes. (a) Normal. (b) stage 1. (c) stage 2. (d) stage 3. (e) stage 4. (f) stage 5.

Fig. 2.

An overview of the proposed ADS-Net based ROP detection framework. The ADS-Net consists of a feature extractor and three classifiers, where two auxiliary classifiers is in red and blue dotted boxes and a master classifier is in green dotted box. In addition, ‘MP’, ‘AAP’, ‘fc’ and ‘softmax’ represent max pooling operator, adaptive average pooling operator, fully connected operator, and Softmax activation layer, while ‘L’ represents multiple stacked dense connection module, ‘P’ represents the predicted classification results and ‘MsFA’ represents the proposed multi-semantic feature aggregation module as shown in Fig. 3.

Fig. 3.

Multi-semantic feature aggregation (MsFA) module. ‘ $F_1$ ’ and ‘ $F_2$ ’ represent two different input features of MsFA module, where ‘ $F_1$ ’ and ‘ $F_2$ ’ come from ‘ $L_3$ ’ and ‘ $L_4$ ’ in Fig. 2, respectively. ‘E’, ‘ $F_T$ ’ and ‘ $F_f$ ’ represent the similarity matrix, spatial response matrix and output feature, respectively. ‘T’ is a transform operation, which is obtained by a 1 × 1 convolution as shown in Eq. (1). In addition, ‘Q’, ‘K’ and ‘V’ are similar to the three branches of self-attention mechanism (query, key and value), which are realized by three 1 × 1 convolutions as shown in Eq. (2), (3) and (4).