Two-stage hybrid network for segmentation of COVID-19 pneumonia lesions in CT images: a multicenter study

Abstract

COVID-19 has been spreading continuously since its outbreak, and the detection of its manifestations in the lung via chest computed tomography (CT) imaging is essential to investigate the diagnosis and prognosis of COVID-19 as an indispensable step. Automatic and accurate segmentation of infected lesions is highly required for fast and accurate diagnosis and further assessment of COVID-19 pneumonia. However, the two-dimensional methods generally neglect the intraslice context, while the three-dimensional methods usually have high GPU memory consumption and calculation cost. To address these limitations, we propose a two-stage hybrid UNet to automatically segment infected regions, which is evaluated on the multicenter data obtained from seven hospitals. Moreover, we train a 3D-ResNet for COVID-19 pneumonia screening. In segmentation tasks, the Dice coefficient reaches 97.23% for lung segmentation and 84.58% for lesion segmentation. In classification tasks, our model can identify COVID-19 pneumonia with an area under the receiver-operating characteristic curve value of 0.92, an accuracy of 92.44%, a sensitivity of 93.94%, and a specificity of 92.45%. In comparison with other state-of-the-art methods, the proposed approach could be implemented as an efficient assisting tool for radiologists in COVID-19 diagnosis from CT images.

Keywords: COVID-19; Computed tomography; Infected lesion segmentation; Screening.

Fig. 1

Main framework of the proposed model. The structure of TSH-UNet for lung and lesion segmentation includes a 2D network and a 3D network, the detailed architecture of which can be found in supplementary Fig. 1 and supplementary Table 1. The input volume data are transformed into three consecutive slices and fed into the 2D network, yielding a coarse segmentation result. Then, after concatenation with the predicted volumes from 2D network, the input volume data are fed into 3D network to extract interslice features. Finally, the hybrid features are jointly optimized in the 3D network to accurately segment lungs and lesions. For classification, the region of interest is extracted using the segmentation result. The 3D network used in the classification part of our model is adapted from 3D ResNet. Finally, the model outputs the probability of COVID-19 pneumonia and common pneumonia. The 2D network consisted of 167 layers, including convolutional layers, pooling layers, upsampling layers, transition layers, and a dense block. The dense block represented the cascade of several microblocks, where all layers were directly connected. The transition layers were used to resize the feature maps, which were composed of a batch normalization layer and convolutional layer (1 × 1) followed by an average pooling layer. We set the compression factor as 0.5 in the transition layers to prevent the feature maps from expanding. Bilinear interpolation was employed in the upsampling layer, followed by the sum of low-level features and a convolutional layer (3 × 3). In addition, batch normalization and the rectified linear unit were used before each convolutional layer in the architecture

Fig. 2

a Raw CT image: the red and green lines represent the outlines of lung and lesions, respectively. b Ground-truth label: the black, red, and green regions represent background, lung, and lesions, respectively.

Fig. 3

Representative segmentation results of testing dataset obtained from different segmentation networks. We selected one typical slice as an example to elucidate the difference between results

Fig. 4

Representative segmentation results of the online datasets obtained from different segmentation networks. We selected one typical slice as an example to elucidate the difference between results

Fig. 5

Representative segmentation results of different centers. We selected one typical slice from the datasets as an example

Fig. 6

ROC curves and confusion matrixes of the classification model using results of different segmentation network on the testing datasets

Fig. 7

Classification performance of the model on datasets obtained from AMU, BYH, and ZRH