A Cascade-SEME network for COVID-19 detection in chest x-ray images

Abstract

Purpose: The worldwide spread of the SARS-CoV-2 virus poses unprecedented challenges to medical resources and infection prevention and control measures around the world. In this case, a rapid and effective detection method for COVID-19 can not only relieve the pressure of the medical system but find and isolate patients in time, to a certain extent, slow down the development of the epidemic. In this paper, we propose a method that can quickly and accurately diagnose whether pneumonia is viral pneumonia, and classify viral pneumonia in a fine-grained way to diagnose COVID-19.

Methods: We proposed a Cascade Squeeze-Excitation and Moment Exchange (Cascade-SEME) framework that can effectively detect COVID-19 cases by evaluating the chest x-ray images, where SE is the structure we designed in the network which has attention mechanism, and ME is a method for image enhancement from feature dimension. The framework integrates a model for a coarse level detection of virus cases among other forms of lung infection, and a model for fine-grained categorisation of pneumonia types identifying COVID-19 cases. In addition, a Regional Learning approach is proposed to mitigate the impact of non-lesion features on network training. The network output is also visualised, highlighting the likely areas of lesion, to assist experts' assessment and diagnosis of COVID-19.

Results: Three datasets were used: a set of Chest x-ray Images for Classification with bacterial pneumonia, viral pneumonia and normal chest x-rays, a COVID chest x-ray dataset with COVID-19, and a Lung Segmentation dataset containing 1000 chest x-rays with masks in the lung region. We evaluated all the models on the test set. The results shows the proposed SEME structure significantly improves the performance of the models: in the task of pneumonia infection type diagnosis, the sensitivity, specificity, accuracy and F1 score of ResNet50 with SEME structure are significantly improved in each category, and the accuracy and AUC of the whole test set are also enhanced; in the detection task of COVID-19, the evaluation results shows that when SEME structure was added to the task, the sensitivities, specificities, accuracy and F1 scores of ResNet50 and DenseNet169 are improved. Although the sensitivities and specificities are not significantly promoted, SEME well balanced these two significant indicators. Regional learning also plays an important role. Experiments show that Regional Learning can effectively correct the impact of non-lesion features on the network, which can be seen in the Grad-CAM method.

Conclusions: Experiments show that after the application of SEME structure in the network, the performance of SEME-ResNet50 and SEME-DenseNet169 in both two datasets show a clear enhancement. And the proposed regional learning method effectively directs the network's attention to focus on relevant pathological regions in the lung radiograph, ensuring the performance of the proposed framework even when a small training set is used. The visual interpretation step using Grad-CAM finds that the region of attention on radiographs of different types of pneumonia are located in different regions of the lungs.

Keywords: COVID-19; Cascade-SEMEnet; chest x-rays; grad-CAM; regional learning.

Fig. 1

Display of dataset1 and dataset2.

Fig. 2

A new cascade network structure with the most potential. (a) is the MoEx structures, (b) is the SE structures, (c) is the Global Average Pooling (GAP) layers, and (d) is the process of Regional Learning.

Fig. 3

An illustration of the MoEx component, its feature extraction process (a) and its role in the network (b).

Fig. 4

Illustrations of (a) the Squeeze‐and‐Exception (SE) structure, (b) the SE‐ResNet structure and (c) the SE‐DenseNet structure.

Fig. 5

Resize operation leads to loss of lesion details in the image.

Fig. 6

(a) A schematic diagram of GAP, (b) the receptive field area before the GAP structure is increased, (c) the receptive field area after the GAP structure is increased.

Fig. 7

When the network identifies a non‐target area as the basis for decision, Regional Learning uses the corresponding Target Region to guide the network to focus on a true target area.

Fig. 8

(a), (b), (c) training curves on Dataset1. (d) training curves on Dataset2. M1: Base‐VGG19, M2: Base‐ResNet50, M3: Base‐DenseNet169, M4: VGG19, M5: ResNet50, M6: DenseNet169, M7: SE‐ResNet50, M8: SE‐DenseNet169, M9: SEME‐ResNet50, M10: SEME‐DenseNet169, M11: MoEx‐ResNet50, M12: MoEx‐DenseNet169, M13: SEME‐ResNet50, M14: SEME‐DenseNet169. line: Training curves, dots: Validation curves.

Fig. 9

ROC curves of different models and different Datasets. (a) (b) (c) ROC curves of Dataset1, (d) ROC curves of Dataset2. M1: Base‐VGG19, M2: Base‐ResNet50, M3: Base‐DenseNet169, M4: VGG19, M5: ResNet50, M6: DenseNet169, M7: SE‐ResNet50, M8: SE‐DenseNet169, M9: SEME‐ResNet50, M10: SEME‐DenseNet169, M11: MoEx‐ResNet50, M12: MoEx‐DenseNet169, M13: SEME‐ResNet50, M14: SEME‐DenseNet169.

Fig. 10

The confusion matrices of Models in Dataset1. M1: Base‐VGG19, M2: Base‐ResNet50, M3: Base‐DenseNet169, M4: VGG19, M5: ResNet50, M6: DenseNet169, M7: SE‐ResNet50, M8: SE‐DenseNet169, M9: SEME‐ResNet50, M10: SEME‐DenseNet169.

Fig. 11

The result of data visualization in Dataset1 by Grad‐CAM.

Fig. 12

The significance test of models in Dataset1. M1: Base‐VGG19, M2: Base‐ResNet50, M3: Base‐DenseNet169, M4: VGG19, M5: ResNet50, M6: DenseNet169, M7: SE‐ResNet50, M8: SE‐DenseNet169, M9: SEME‐ResNet50, M10: SEME‐DenseNet169.

Fig. 13

(a) Grad‐CAM based visualization result of the VGG19 network trained on dataset2 original data, (b) the corresponding training and validation curves.

Fig. 14

(a) is the loss and mean IOU in the verification set during the U‐Net training, (b) flow chart of Regional Learning.

Fig. 15

The confusion matrices of Models in Dataset2. M11: MoEx‐ResNet50, M12: MoEx‐DenseNet169, M13: SEME‐ResNet50, M14: SEME‐DenseNet169.

Fig. 16

The result of data visualization in Dataset2 by Grad‐CAM.

Fig. 17

The significance test of models in Dataset2.

Fig. 18

Characteristic distribution in two different angles of chest radiograph focus areas in bacterial pneumonia, COVID‐19 and other viral pneumonia.