Enhanced nuclear information fusion and visual transformer for pathological breast cancer image classification

Abstract

Breast cancer poses a significant threat to women's health. Early diagnosis using pathological images is crucial for effective treatment planning. However, the low resolution of pathological images poses significant challenges for the extraction of valid information, while their high complexity greatly increases the difficulty of image analysis. To address these challenges, this paper introduces an innovative classification method for breast cancer histopathological images, combining enhanced nuclear information with an Enhanced Vision Transformer (EVT) model using wavelet position embedding. The quintessence of the proposed method resides in its capacity to efficiently extract both biological and foundational image features from pathological images. This is accomplished by initially enhancing nuclear information through the application of segmentation models and sophisticated image processing techniques. Subsequently, wavelet positional embedding within the EVT model is leveraged to precisely capture key information embedded within the images. Experimental outcomes have demonstrated that our method attains an accuracy rate of 94.61% and an AUC value of 99.07% on the BreaKHis dataset, significantly outperforming other baseline network models in terms of classification efficacy. Furthermore, through visual representation, this study underscores the significance of nuclear information enhancement and wavelet position transformation in the EVT model, thereby further confirming the effectiveness and effectiveness of the method we proposed.

Keywords: Classification; Enhanced nuclear information fusion; Pathological breast cancer image; Segmentation; Visual transformer.

Fig. 1

Shows the comparison of details, with raw data on the left and enhanced data on the right. (a) Differences before and after image enhancement. (b) The difference of channel 3 after normalization before and after image enhancement.

Fig. 2

The overall flow of the method. The framework consists of three main stages: (1) Nuclear information acquisition, where the BreakHis dataset is processed by a segmentation model (2) Enhanced dataset construction, where the segmented nuclei are refined via scale-adjusted fusion and (3) Classification task implementation, where the enhanced data is.

Fig. 3

Schematic diagram of nuclear fusion method. The pipeline processes histopathology images through nucleus segmentation, canny edge detection and edge strengthening.

Fig. 4

EVT structure. The network processes input through tokenization blocks (convolution/ReLU/pooling), parallel branches (transpose/FC/attention), and a classifier with wavelet position embedding, ultimately outputting binary classification results.

Fig. 5

Comparison of original data, nucleus segmentation results, and enhanced data for benign and malignant cases.

Fig. 6

Confusion matrices comparing test data performance of EVT models with nuclear information enhancement and wavelet position transform.

Fig. 7

ROC curves and AUC values illustrating the classification performance of various EVT model combinations on test data.

Fig. 8

The color map is displayed after normalizing the tensor before the last flattener operation in the tokenizer.

Fig. 9

Network model visualization. The a, c, e, g and i is the model.tokenizer.conv_layers[0][0] layer, the first convolutional layer in the network. The b, d, f, h and j is the Model.Tokenizer.conv_layers[2][2] layer, which is the last maximum pooling layer of the tokenizer module.