Mixed pyramid attention network for nuclear cataract classification based on anterior segment OCT images

Abstract

Nuclear cataract (NC) is a leading ocular disease globally for blindness and vision impairment. NC patients can improve their vision through cataract surgery or slow the opacity development with early intervention. Anterior segment optical coherence tomography (AS-OCT) image is an emerging ophthalmic image type, which can clearly observe the whole lens structure. Recently, clinicians have been increasingly studying the correlation between NC severity levels and clinical features from the nucleus region on AS-OCT images, and the results suggested the correlation is strong. However, automatic NC classification research based on AS-OCT images has rarely been studied. This paper presents a novel mixed pyramid attention network (MPANet) to classify NC severity levels on AS-OCT images automatically. In the MPANet, we design a novel mixed pyramid attention (MPA) block, which first applies the group convolution method to enhance the feature representation difference of feature maps and then construct a mixed pyramid pooling structure to extract local-global feature representations and different feature representation types simultaneously. We conduct extensive experiments on a clinical AS-OCT image dataset and a public OCT dataset to evaluate the effectiveness of our method. The results demonstrate that our method achieves competitive classification performance through comparisons to state-of-the-art methods and previous works. Moreover, this paper also uses the class activation mapping (CAM) technique to improve our method's interpretability of classification results.

Keywords: AS-OCT images; CNN; Classification; Mixed pyramid attention; Nuclear cataract.

Fig. 1

Three nuclear cataract severity levels based on AS-OCT images (a). Mild nuclear cataract (b) with slight opacity but is asymptomatic. Moderate nuclear cataract (c) with moderate opacity and is symptomatic. Severe nuclear cataract (d) with severe opacity and is symptomatic obviously

Fig. 2

The architecture of the Mixed Pyramid Attention Network (MPANet). We devise the MPA block by integrating clinical prior knowledge, and then utilize the MPA block to construct the Residual-MPA module by plugging it into the Residual module. MPANet (a) is used for NC classification by using the nucleus region from AS-OCT images, comprised of multiple Residual-MPA modules. We use a deep CNN model to acquire the nucleus region automatically. MPA block comprises a group convolution layer, a mixed pyramid pooling structure, and multi-layer perception. Green and blue pointwise convolutions (Conv. $1\times 1$) denote two learned feature representation types

Fig. 3

Toy example comparisons between standard convolution method and group convolution method

Fig. 4

Performance comparison of our MPANet and strong baselines in sensitivity and F1

Fig. 5

The confusion matrix of MPANet

Fig. 6

The CAM visualization results of our proposed MPANet and other state-of-the-art attention-based CNNs. Row 1 to row 3 denotes the mild NC, moderate NC, and severe NC. The heat maps highlight the informative regions that CNNs learned for specific NC severity levels