Computer-aided diagnostic for classifying chest X-ray images using deep ensemble learning

Abstract

Background: Nowadays doctors and radiologists are overwhelmed with a huge amount of work. This led to the effort to design different Computer-Aided Diagnosis systems (CAD system), with the aim of accomplishing a faster and more accurate diagnosis. The current development of deep learning is a big opportunity for the development of new CADs. In this paper, we propose a novel architecture for a convolutional neural network (CNN) ensemble for classifying chest X-ray (CRX) images into four classes: viral Pneumonia, Tuberculosis, COVID-19, and Healthy. Although Computed tomography (CT) is the best way to detect and diagnoses pulmonary issues, CT is more expensive than CRX. Furthermore, CRX is commonly the first step in the diagnosis, so it's very important to be accurate in the early stages of diagnosis and treatment.

Results: We applied the transfer learning technique and data augmentation to all CNNs for obtaining better performance. We have designed and evaluated two different CNN-ensembles: Stacking and Voting. This system is ready to be applied in a CAD system to automated diagnosis such a second or previous opinion before the doctors or radiology's. Our results show a great improvement, 99% accuracy of the Stacking Ensemble and 98% of accuracy for the the Voting Ensemble.

Conclusions: To minimize missclassifications, we included six different base CNN models in our architecture (VGG16, VGG19, InceptionV3, ResNet101V2, DenseNet121 and CheXnet) and it could be extended to any number as well as we expect extend the number of diseases to detected. The proposed method has been validated using a large dataset created by mixing several public datasets with different image sizes and quality. As we demonstrate in the evaluation carried out, we reach better results and generalization compared with previous works. In addition, we make a first approach to explainable deep learning with the objective of providing professionals more information that may be valuable when evaluating CRXs.

Keywords: CNN; COVID-19 classification; Deep ensemble learning; Grad-CAM; Stacking; Voting.

Fig. 1

Proposed architecture of computer-aided diagnosis system based on CNN-ensemble

Fig. 2

CRX examples from the dataset

Fig. 3

Stacking Ensemble architecture

Fig. 4

Receiver operating characteristic (ROC) and precision-recall curve (PR) for the CNN (micro-averaging)

Fig. 5

Grad-CAM heatmaps for pneumonia CRX images

Fig. 6

Grad-CAM heatmaps for tuberculosis CRX images with radiological findings

Fig. 7

Validation confusion matrix (classes: 0-COVID-19, 1-Healthy, 2-Pneumonia, and 3-Tuberculosis)

Fig. 8

Training/validation loss and accuracy graph for the stacking ensemble

Fig. 9

Test confusion matrix (classes: 0-COVID-19, 1-Healthy, 2-Pneumonia, and 3-Tuberculosis)

Fig. 10

TSNE visualization for train and validation set. a Originals radiography images, b intermediate layer VGG19 (block3—pool), and c last layer VGG19 (dense layer). Classes: Blue-COVID-19, Green-Healthy, Purple-Pneumonia, and Orange-Tuberculosis

Fig. 11

Receiver operating characteristic (ROC) for stacking ensemble (a) and voting ensemble (b) (classes: 0-COVID-19, 1-Healthy, 2-Pneumonia, and 3-Tuberculosis)

Fig. 12

Precision-recall curve for stacking ensemble (a) and voting ensemble (b) (classes: 0-COVID-19, 1-Healthy, 2-Pneumonia, and 3-Tuberculosis)