FSS-2019-nCov: A deep learning architecture for semi-supervised few-shot segmentation of COVID-19 infection

Abstract

The newly discovered coronavirus (COVID-19) pneumonia is providing major challenges to research in terms of diagnosis and disease quantification. Deep-learning (DL) techniques allow extremely precise image segmentation; yet, they necessitate huge volumes of manually labeled data to be trained in a supervised manner. Few-Shot Learning (FSL) paradigms tackle this issue by learning a novel category from a small number of annotated instances. We present an innovative semi-supervised few-shot segmentation (FSS) approach for efficient segmentation of 2019-nCov infection (FSS-2019-nCov) from only a few amounts of annotated lung CT scans. The key challenge of this study is to provide accurate segmentation of COVID-19 infection from a limited number of annotated instances. For that purpose, we propose a novel dual-path deep-learning architecture for FSS. Every path contains encoder-decoder (E-D) architecture to extract high-level information while maintaining the channel information of COVID-19 CT slices. The E-D architecture primarily consists of three main modules: a feature encoder module, a context enrichment (CE) module, and a feature decoder module. We utilize the pre-trained ResNet34 as an encoder backbone for feature extraction. The CE module is designated by a newly introduced proposed Smoothed Atrous Convolution (SAC) block and Multi-scale Pyramid Pooling (MPP) block. The conditioner path takes the pairs of CT images and their labels as input and produces a relevant knowledge representation that is transferred to the segmentation path to be used to segment the new images. To enable effective collaboration between both paths, we propose an adaptive recombination and recalibration (RR) module that permits intensive knowledge exchange between paths with a trivial increase in computational complexity. The model is extended to multi-class labeling for various types of lung infections. This contribution overcomes the limitation of the lack of large numbers of COVID-19 CT scans. It also provides a general framework for lung disease diagnosis in limited data situations.

Keywords: COVID-19; CT images; Context fusion; Deep learning; Few-shot segmentation.

Fig. 1

The architecture of the proposed FSS-2019-nCov. It consists of two identical paths with the encoder–decoder structure, namely the conditioner path (upper) and the segmentation path (top). The recombination and recalibration (RR) blocks (see Fig. 4) are introduced to effectuate knowledge interaction between two paths. The axial CT images are passed through a feature encoder blocks (E) module that is implemented with the pre-trained ResNet-34 blocks. The context enrichment module is then introduced to generate an improved semantic representation using SAC and MPP modules. Finally, the acquired representations passed into the feature decoder blocks (D).

Fig. 2

Illustration of the encoder and decoder modules used in the proposed FSS-2019-nCov: (a) the encoder module implemented using Res2Net module ; and (b) the architecture of the decoder module.

Fig. 3

The architecture of the proposed SAC module consisting of four parallel paths. Each path from left to right contains 1, 2, 3, and 4 separable and shared convolutions, respectively.

Fig. 4

The architecture of the proposed MPP module containing five parallel paths for changing input resolution. Convolution layers are employed to capture different resolution information. The global average pooling (GAP) layer is employed to implement the residual connection.

Fig. 5

The architecture of the RR module: (a) illustration of the recalibration block implemented using separable and shareable convolution; (b) illustration of the recombination block; and (c) integration of both recalibration and recombination in a single module.

Fig. 6

Lung infection segmentation using proposed FSS-2019-nCov. The first row represents the original CT image from the test set. The corresponding segmentation outcome from the U-Netv , U-Net++ , Inf-Net , Semi-Inf-Net , SE-Net  are presented in the second, third, fourth, fifth, sixth row respectively. The segmentation results of the proposed FSS-2019-nCov is presented in the seventh row. The corresponding ground truth label for every image is presented at the bottom of the last row of images.