RGSB-UNet: Hybrid Deep Learning Framework for Tumour Segmentation in Digital Pathology Images

Abstract

Colorectal cancer (CRC) is a prevalent gastrointestinal tumour with high incidence and mortality rates. Early screening for CRC can improve cure rates and reduce mortality. Recently, deep convolution neural network (CNN)-based pathological image diagnosis has been intensively studied to meet the challenge of time-consuming and labour-intense manual analysis of high-resolution whole slide images (WSIs). Despite the achievements made, deep CNN-based methods still suffer from some limitations, and the fundamental problem is that they cannot capture global features. To address this issue, we propose a hybrid deep learning framework (RGSB-UNet) for automatic tumour segmentation in WSIs. The framework adopts a UNet architecture that consists of the newly-designed residual ghost block with switchable normalization (RGS) and the bottleneck transformer (BoT) for downsampling to extract refined features, and the transposed convolution and 1 × 1 convolution with ReLU for upsampling to restore the feature map resolution to that of the original image. The proposed framework combines the advantages of the spatial-local correlation of CNNs and the long-distance feature dependencies of BoT, ensuring its capacity of extracting more refined features and robustness to varying batch sizes. Additionally, we consider a class-wise dice loss (CDL) function to train the segmentation network. The proposed network achieves state-of-the-art segmentation performance under small batch sizes. Experimental results on DigestPath2019 and GlaS datasets demonstrate that our proposed model produces superior evaluation scores and state-of-the-art segmentation results.

Keywords: Residual-Ghost-SN; bottleneck transformer; hybrid deep learning framework; tumour segmentation; whole slide image.

Figure 1

An overview of RGSB-UNet. The TRCCR denotes transposed convolution, ReLU, concatenate, convolution, and ReLU.

Figure 2

Schematic diagram of RGSB-UNet. RGS denotes the proposed residual ghost block with switchable normalization, and BoT denotes the bottleneck transformer. MP and AP denote the max and average pooling, respectively. Tconv denotes the transposed convolution used for upsampling.

Figure 3

Schematic diagram of Ghost block with switchable normalization. The dash box denotes the cheap operation that uses a 3 × 3 group convolution in the ghost block.

Figure 4

Schematic diagram of the proposed bottleneck. (a) RGS Bottleneck. (b) Bottleneck transformer. GBS and SN denote the ghost block with switchable normalization and switchable normalization, respectively. MHSA denotes multi-head self-attention.

Figure 5

Schematic diagram of (a) self-attention [26] and (b) multi-head self-attention.

Figure 6

Samples cropped from WSI.

Figure 7

Segmentation results of different networks on the DigestPath2019 dataset. In the superimposed images, the areas marked in green represent the ground truth.

Figure 8

Segmentation results of different networks on the GlaS dataset. In the superimposed images, the areas marked in green represent the ground truth.