An attention-supervised full-resolution residual network for the segmentation of breast ultrasound images

Abstract

Purpose: Breast cancer is the most common cancer among women worldwide. Medical ultrasound imaging is one of the widely applied breast imaging methods for breast tumors. Automatic breast ultrasound (BUS) image segmentation can measure the size of tumors objectively. However, various ultrasound artifacts hinder segmentation. We proposed an attention-supervised full-resolution residual network (ASFRRN) to segment tumors from BUS images.

Methods: In the proposed method, Global Attention Upsample (GAU) and deep supervision were introduced into a full-resolution residual network (FRRN), where GAU learns to merge features at different levels with attention for deep supervision. Two datasets were employed for evaluation. One (Dataset A) consisted of 163 BUS images with tumors (53 malignant and 110 benign) from UDIAT Centre Diagnostic, and the other (Dataset B) included 980 BUS images with tumors (595 malignant and 385 benign) from the Sun Yat-sen University Cancer Center. The tumors from both datasets were manually segmented by medical doctors. For evaluation, the Dice coefficient (Dice), Jaccard similarity coefficient (JSC), and F1 score were calculated.

Results: For Dataset A, the proposed method achieved higher Dice (84.3 $\pm$ 10.0%), JSC (75.2 $\pm$ 10.7%), and F1 score (84.3 $\pm$ 10.0%) than the previous best method: FRRN. For Dataset B, the proposed method also achieved higher Dice (90.7 $\pm$ 13.0%), JSC (83.7 $\pm$ 14.8%), and F1 score (90.7 $\pm$ 13.0%) than the previous best methods: DeepLabv3 and dual attention network (DANet). For Dataset A + B, the proposed method achieved higher Dice (90.5 $\pm$ 13.1%), JSC (83.3 $\pm$ 14.8%), and F1 score (90.5 $\pm$ 13.1%) than the previous best method: DeepLabv3. Additionally, the parameter number of ASFRRN was only 10.6 M, which is less than those of DANet (71.4 M) and DeepLabv3 (41.3 M).

Conclusions: We proposed ASFRRN, which combined with FRRN, attention mechanism, and deep supervision to segment tumors from BUS images. It achieved high segmentation accuracy with a reduced parameter number.

Keywords: breast cancer; breast ultrasound image; deep learning; segmentation.

FIG. 1.

Conceptual structures of (a) U-Net, (b) FRRN: full-resolution residual network, and (c) ASFRRN: attention-supervised FRRN.

Fig. 2.

Structures of (a) FRRU and (b) FRRU × 4. FRRU: full-resolution residual unit.

Fig. 3.

Structure of ASFRRN (ASFRRN: attention-supervised full-resolution residual network).

Fig. 4.

Structure of GAU (GAU: Global Attention Upsample).

Fig. 5.

Image samples of datasets. The first and second rows show the original BUS images and their ground truth segmentation, respectively. The first two columns are from Dataset A, and the last two columns are from Dataset B.

FIG. 6.

Changes in training and validation errors for the analyzed networks.

Fig. 7.

Segmentation results for different networks (U-Net, ResU-Net, Squeeze U-Net, Attention ResU-Net, DeconvNet, DFCN, DANet, Deeplabv3, FRRN, ASFRRN). The red contour denotes the ground truth, and the yellow contour represents the segmentation results. Images in the first two rows are from Dataset A. Images in the last two rows are from Dataset B.

Fig. 8.

Feature maps of the residual stream after each summation, shown in the order by which the residual stream deepens. The red contour denotes the ground truth. Two randomly chosen channels of each feature map are shown.

Fig. 9.

Feature maps before and after channel-wise attention in the 3rd GAU. (a) Original BUS image; (b) Feature maps of channels 49, 53, 1, 5, 58, 56, 6, and 8 before channel-wise attention: (c) Corresponding feature maps after channel-wise attention. The red contour denotes the ground truth.

Fig. 10.

Feature maps before and after channel-wise attention in the 3rd GAU. (a) Original BUS image; (b) Feature maps of channels 15, 31, 9, 13, 33, 35, 17, and 26 before channel-wise attention: (c) corresponding feature maps after channel-wise attention. The red contour denotes the ground truth.