Covid-MANet: Multi-task attention network for explainable diagnosis and severity assessment of COVID-19 from CXR images

Abstract

The devastating outbreak of Coronavirus Disease (COVID-19) cases in early 2020 led the world to face health crises. Subsequently, the exponential reproduction rate of COVID-19 disease can only be reduced by early diagnosis of COVID-19 infection cases correctly. The initial research findings reported that radiological examinations using CT and CXR modality have successfully reduced false negatives by RT-PCR test. This research study aims to develop an explainable diagnosis system for the detection and infection region quantification of COVID-19 disease. The existing research studies successfully explored deep learning approaches with higher performance measures but lacked generalization and interpretability for COVID-19 diagnosis. In this study, we address these issues by the Covid-MANet network, an automated end-to-end multi-task attention network that works for 5 classes in three stages for COVID-19 infection screening. The first stage of the Covid-MANet network localizes attention of the model to the relevant lungs region for disease recognition. The second stage of the Covid-MANet network differentiates COVID-19 cases from bacterial pneumonia, viral pneumonia, normal and tuberculosis cases, respectively. To improve the interpretation and explainability, three experiments have been conducted in exploration of the most coherent and appropriate classification approach. Moreover, the multi-scale attention model MA-DenseNet201 proposed for the classification of COVID-19 cases. The final stage of the Covid-MANet network quantifies the proportion of infection and severity of COVID-19 in the lungs. The COVID-19 cases are graded into more specific severity levels such as mild, moderate, severe, and critical as per the score assigned by the RALE scoring system. The MA-DenseNet201 classification model outperforms eight state-of-the-art CNN models, in terms of sensitivity and interpretation with lung localization network. The COVID-19 infection segmentation by UNet with DenseNet121 encoder achieves dice score of 86.15% outperforming UNet, UNet++, AttentionUNet, R2UNet, with VGG16, ResNet50 and DenseNet201 encoder. The proposed network not only classifies images based on the predicted label but also highlights the infection by segmentation/localization of model-focused regions to support explainable decisions. MA-DenseNet201 model with a segmentation-based cropping approach achieves maximum interpretation of 96% with COVID-19 sensitivity of 97.75%. Finally, based on class-varied sensitivity analysis Covid-MANet ensemble network of MA-DenseNet201, ResNet50 and MobileNet achieve 95.05% accuracy and 98.75% COVID-19 sensitivity. The proposed model is externally validated on an unseen dataset, yields 98.17% COVID-19 sensitivity.

Keywords: Chest X-ray; Covid-19; Deep learning; Explainable AI; Infection segmentation; Lung segmentation; Transfer learning.

Fig. 1

Example sample images; (a) COVID-19 (b) Normal (c) BP (d) VP (e) TB classes.

Fig. 2

Architecture of lung segmentation network, (a) basic UNet type model where 3 × 3 block is replaced by block b, c or d (b) Convolutional block of UNet (c) Residual convolutional block of ResUNet (d) Dense convolutional block of DenseUNet model.

Fig. 3

Example of segmented CXR samples, (a) COVID-19, (b) Normal, (C) BP, (d) VP, (e) TB, where rows correspond to the original pre-processed image, predicted lung masks, post-processed lung masks and final segmented lung contour.

Fig. 4

The architecture of MA-DenseNet201 model adding multiscale hybrid attention module to DenseNet201 backbone.

Fig. 5

Visualization of Grad-CAM maps for each class, where the first row corresponds to input images. The second, third and fourth rows show Grad-CAM activation visualization produced by the best classification model in each of the experiment indicating proposed segmentation-based cropping approach has better interpretable results.

Fig. 6

The framework of Covid-MANet methodology for classification and infection segmentation of COVID-19.

Fig. 7

Comparison of ground truth and predicted lung masks by the lung segmentation models.

Fig. 8

The first row shows confusion matrix of proposed model and second row representing learning curves in each best model.

Fig. 9

Visualization of ROC map for top four performing models representing area under curve for each of class.

Fig. 10

The sensitivity comparison of models where first row shows result of class-wise sensitivity analysis by each of model in 1st^,2nd and 3rd experiment and the second row shows sensitivity comparison at different confidence threshold values. The red marked bracket indicates the proposed model is better in all cases after lung localization approach.

Fig. 11

The qualitative comparison of ground truth masks to the masks predicted by infection segmentation models where column 1 shows ground truth infection mask and column 2–7 shows masks predicted by infection segmentation models, respectively.

Fig. 12

The infection segmentation and severity grading results by the proposed model on COVID-19 classified samples graded as mild, moderate, severe, and critical, where the first row shows both lung and infection mask contours giving intuition for infection quantification.

Fig. 13

Interpretation of disease map focused using Grad-CAM for each disease type by Covid-MANet improved after supervision by segmentation-based cropping.