4S-DT: Self-Supervised Super Sample Decomposition for Transfer Learning With Application to COVID-19 Detection

Abstract

Due to the high availability of large-scale annotated image datasets, knowledge transfer from pretrained models showed outstanding performance in medical image classification. However, building a robust image classification model for datasets with data irregularity or imbalanced classes can be a very challenging task, especially in the medical imaging domain. In this article, we propose a novel deep convolutional neural network, which we called self-supervised super sample decomposition for transfer learning (4S-DT) model. The 4S-DT encourages a coarse-to-fine transfer learning from large-scale image recognition tasks to a specific chest X-ray image classification task using a generic self-supervised sample decomposition approach. Our main contribution is a novel self-supervised learning mechanism guided by a super sample decomposition of unlabeled chest X-ray images. 4S-DT helps in improving the robustness of knowledge transformation via a downstream learning strategy with a class-decomposition (CD) layer to simplify the local structure of the data. The 4S-DT can deal with any irregularities in the image dataset by investigating its class boundaries using a downstream CD mechanism. We used 50000 unlabeled chest X-ray images to achieve our coarse-to-fine transfer learning with an application to COVID-19 detection, as an exemplar. The 4S-DT has achieved a high accuracy of 99.8% on the larger of the two datasets used in the experimental study and an accuracy of 97.54% on the smaller dataset, which was enriched by augmented images, out of which all real COVID-19 cases were detected.

Fig. 1.

Examples of (a) labeled chest X-ray images (from left to right: normal, COVID-19, and SARS images) and (b) unlabeled chest X-ray images used in this work for self-supervision learning.

Fig. 2.

Graphical representation of 4S-DT model. In stage I, a self-supervised sample decomposition mechanism has been designed to generate pseudo labels for unlabeled chest X-ray images. Then, in stage II, the pseudo labels are used to achieve coarse transfer learning, resulting in chest X-ray-related deep features. Finally, in stage III, a downstream training has been accomplished by adapting a fine transfer learning from chest X-ray recognition (achieved by the pretext training of stage II) to COVID-19 detection.

Fig. 3.

Example of a reconstructed chest X-ray image by our AE. (a) Original image. (b) Reconstructed image.

Fig. 4.

Estimation of the optimal Eps.

Fig. 5.

Confusion matrix obtained by (a) 4S-DT on the 58 test set and (b) 4S-DT when trained on augmented images only and tested on the 196 cases of (COVID-19 dataset-A).

Fig. 6.

Confusion matrix results of COVID-19 dataset-B obtained by 4S-DT, using different pretrained networks: (a) ResNet18, (b) GoogleNet, and (c) VGG19.

Fig. 7.

ROC curve obtained during the training of 4S-DT on the COVID-19 dataset-A and COVID-19 dataset-B.