MultiR-Net: A Novel Joint Learning Network for COVID-19 segmentation and classification

Abstract

The outbreak of COVID-19 has caused a severe shortage of healthcare resources. Ground Glass Opacity (GGO) and consolidation of chest CT scans have been an essential basis for imaging diagnosis since 2020. The similarity of imaging features between COVID-19 and other pneumonia makes it challenging to distinguish between them and affects radiologists' diagnosis. Recently, deep learning in COVID-19 has been mainly divided into disease classification and lesion segmentation, yet little work has focused on the feature correlation between the two tasks. To address these issues, in this study, we propose MultiR-Net, a 3D deep learning model for combined COVID-19 classification and lesion segmentation, to achieve real-time and interpretable COVID-19 chest CT diagnosis. Precisely, the proposed network consists of two subnets: a multi-scale feature fusion UNet-like subnet for lesion segmentation and a classification subnet for disease diagnosis. The features between the two subnets are fused by the reverse attention mechanism and the iterable training strategy. Meanwhile, we proposed a loss function to enhance the interaction between the two subnets. Individual metrics can not wholly reflect network effectiveness. Thus we quantify the segmentation results with various evaluation metrics such as average surface distance, volume Dice, and test on the dataset. We employ a dataset containing 275 3D CT scans for classifying COVID-19, Community-acquired Pneumonia (CAP), and healthy people and segmented lesions in pneumonia patients. We split the dataset into 70% and 30% for training and testing. Extensive experiments showed that our multi-task model framework obtained an average recall of 93.323%, an average precision of 94.005% on the classification test set, and a 69.95% Volume Dice score on the segmentation test set of our dataset.

Keywords: COVID-19; Interpretability; Multi-scale; Multi-task learning; V-Net.

Fig. 1

The example of COVID-19 lesions, where the red and green areas denote the consolidation and GGO respectively.

Fig. 2

The overall architecture of MultiR-Net.

Fig. 3

The architecture reverse attention module.

Fig. 4

The diagram for calculating $ASO$.

Fig. 5

The training loss curves of the networks.

Fig. 6

The segmentation performance of the networks.

Fig. 7

The visualization images of 3D and 2D lesion segmentation results.

Fig. 8

The Bland-Altman plots of the segmentation networks.

Fig. 9

The classification performance of the networks.

Fig. 10

The ROC curves of our MultiR-Net and other models of (a) COVID-19, (b) CAP and (c) Normal people.

Fig. 11

The interpretable visualizations of Grad-CAM.

Fig. 12

The effect of Reserve-attention module, Focal Tversky Loss and Iterative Learning Strategy on the performance of segmentation and classification.