CCT: Lightweight compact convolutional transformer for lung disease CT image classification

Abstract

Computed tomography (CT) imaging results are an important criterion for the diagnosis of lung disease. CT images can clearly show the characteristics of lung lesions. Early and accurate detection of lung diseases helps clinicians to improve patient care effectively. Therefore, in this study, we used a lightweight compact convolutional transformer (CCT) to build a prediction model for lung disease classification using chest CT images. We added a position offset term and changed the attention mechanism of the transformer encoder to an axial attention mechanism module. As a result, the classification performance of the model was improved in terms of height and width. We show that the model effectively classifies COVID-19, community pneumonia, and normal conditions on the CC-CCII dataset. The proposed model outperforms other comparable models in the test set, achieving an accuracy of 98.5% and a sensitivity of 98.6%. The results show that our method achieves a larger field of perception on CT images, which positively affects the classification of CT images. Thus, the method can provide adequate assistance to clinicians.

Keywords: COVID-19; axial attention; compact convolutional transformer; image classification; positional bias term.

FIGURE 1

Our compact convolutional transformer models.

FIGURE 2

CCT diagram of convolution module.

FIGURE 3

Encoder structure drawing.

FIGURE 4

Axial attention.

FIGURE 5

CC-CCII chest CT images. (A) Normal conditions; (B) COVID-19; (C) CP.

FIGURE 6

Example of ablation experiment comparison. (A) The CT image of the COVID-19 patient; (B) the result.

FIGURE 7

Confusion matrix. (A) Efficientnet-b7; (B) Mobilenet-v3; (C) ViT; (D) CCT; (E) Ours.