Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Nov:126:104037.
doi: 10.1016/j.compbiomed.2020.104037. Epub 2020 Oct 8.

Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation

Amine Amyar  1 , Romain Modzelewski  2 , Hua Li  3 , Su Ruan  4
Affiliations

Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation

Amine Amyar et al. Comput Biol Med. 2020 Nov.

Abstract

This paper presents an automatic classification segmentation tool for helping screening COVID-19 pneumonia using chest CT imaging. The segmented lesions can help to assess the severity of pneumonia and follow-up the patients. In this work, we propose a new multitask deep learning model to jointly identify COVID-19 patient and segment COVID-19 lesion from chest CT images. Three learning tasks: segmentation, classification and reconstruction are jointly performed with different datasets. Our motivation is on the one hand to leverage useful information contained in multiple related tasks to improve both segmentation and classification performances, and on the other hand to deal with the problems of small data because each task can have a relatively small dataset. Our architecture is composed of a common encoder for disentangled feature representation with three tasks, and two decoders and a multi-layer perceptron for reconstruction, segmentation and classification respectively. The proposed model is evaluated and compared with other image segmentation techniques using a dataset of 1369 patients including 449 patients with COVID-19, 425 normal ones, 98 with lung cancer and 397 of different kinds of pathology. The obtained results show very encouraging performance of our method with a dice coefficient higher than 0.88 for the segmentation and an area under the ROC curve higher than 97% for the classification.

Keywords: Computed tomography images; Coronavirus (COVID-19); Deep learning; Image classification; Image segmentation; Multitask learning.

PubMed Disclaimer

Conflict of interest statement

None Declared.

Figures

Fig. 1
Fig. 1
An example of exams heterogeneities between different CT images for COVID (upper) and non-COVID (bottom). Patient images do not have the same resolution. Also, images show different image formats (Dicom(A), png (B C E F), Nifti (D).
Fig. 2
Fig. 2
Example of COVID-19 segmentation on CT scan for 2 patients: first column - original CT scanner; Second column - one label segmentation, Third column - 3 labels segmentation: ground glass (green), consolidation (blue) and pleural effusion (yellow).
Fig. 3
Fig. 3
The different databases used in this study.
Fig. 4
Fig. 4
Hard parameter sharing for multi-task learning in deep neural networks used in our proposed architecture.
Fig. 5
Fig. 5
Our proposed architecture, composed of an encoder and two decoders for image reconstruction and infection segmentation. A fully connected layers are added for classification (COVID vs Normal vs Other Infections classification).
Fig. 6
Fig. 6
Learning curve of our proposed model. Left is the model loss and right is the model accuracy per epoch.
Fig. 7
Fig. 7
A comparison between our model and U-NET for infection segmentation. From left to right columns: input images, ground truth, results of U-NET, results of our method.
Fig. 8
Fig. 8
ROC curve of Experiment 1 for COVID-19 classification.
Fig. 9
Fig. 9
ROC curve of Experiment 3 for COVID-19 classification.
Fig. 10
Fig. 10
Confusion matrix of our model.
See this image and copyright information in PMC

References

    1. Abbasian Ardakani A., Bitarafan-Rajabi A., Mohammadzadeh A., Mohammadi A., Riazi R., Abolghasemi J., Homayoun Jafari A., Bagher Shiran M. A hybrid multilayer filtering approach for thyroid nodule segmentation on ultrasound images. J. Ultrasound Med. 2019;38:629–640. - PubMed
    1. Afshar P., Heidarian S., Naderkhani F., Oikonomou A., Plataniotis K.N., Mohammadi A. 2020. Covid-caps: A Capsule Network-Based Framework for Identification of Covid-19 Cases from X-Ray Images. arXiv preprint arXiv:2004.02696. - PMC - PubMed
    1. Amyar A., Ruan S., Gardin I., Chatelain C., Decazes P., Modzelewski R. 3-d rpet-net: development of a 3-d pet imaging convolutional neural network for radiomics analysis and outcome prediction. IEEE Transactions on Radiation and Plasma Medical Sciences. 2019;3:225–231.
    1. Amyar A., Ruan S., Gardin I., Herault R., Clement C., Decazes P., Modzelewski R. Radiomics-net: convolutional neural networks on fdg pet images for predicting cancer treatment response. J. Nucl. Med. 2018;59 324–324.
    1. Badrinarayanan V., Kendall A., Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017;39:2481–2495. - PubMed