Pneumonia detection in chest X-ray images using an ensemble of deep learning models

Abstract

Pneumonia is a respiratory infection caused by bacteria or viruses; it affects many individuals, especially in developing and underdeveloped nations, where high levels of pollution, unhygienic living conditions, and overcrowding are relatively common, together with inadequate medical infrastructure. Pneumonia causes pleural effusion, a condition in which fluids fill the lung, causing respiratory difficulty. Early diagnosis of pneumonia is crucial to ensure curative treatment and increase survival rates. Chest X-ray imaging is the most frequently used method for diagnosing pneumonia. However, the examination of chest X-rays is a challenging task and is prone to subjective variability. In this study, we developed a computer-aided diagnosis system for automatic pneumonia detection using chest X-ray images. We employed deep transfer learning to handle the scarcity of available data and designed an ensemble of three convolutional neural network models: GoogLeNet, ResNet-18, and DenseNet-121. A weighted average ensemble technique was adopted, wherein the weights assigned to the base learners were determined using a novel approach. The scores of four standard evaluation metrics, precision, recall, f1-score, and the area under the curve, are fused to form the weight vector, which in studies in the literature was frequently set experimentally, a method that is prone to error. The proposed approach was evaluated on two publicly available pneumonia X-ray datasets, provided by Kermany et al. and the Radiological Society of North America (RSNA), respectively, using a five-fold cross-validation scheme. The proposed method achieved accuracy rates of 98.81% and 86.85% and sensitivity rates of 98.80% and 87.02% on the Kermany and RSNA datasets, respectively. The results were superior to those of state-of-the-art methods and our method performed better than the widely used ensemble techniques. Statistical analyses on the datasets using McNemar's and ANOVA tests showed the robustness of the approach. The codes for the proposed work are available at https://github.com/Rohit-Kundu/Ensemble-Pneumonia-Detection.

Fig 1. Examples of two X-ray plates that display (a) a healthy lung and (b) a pneumonic lung.

The red arrows in (b) indicate white infiltrates, a distinguishing feature of pneumonia. The images were taken from the Kermany dataset [4].

Fig 2. Representation of the proposed pneumonia detection framework.

Pre = Precision score, Rec = Recall score, F1 = F1-score, AUC = AUC score, and A⁽ⁱ⁾ = {Pre_i, Rec_i, F1_i, AUC_i}; w⁽ⁱ⁾ is the weight generated for the i^th base learner to compute the ensemble, $p_j^{\left(i\right)}$ is the probability score for the j^th sample by the i^th classifier, and ens_j is the fused probability score for the j^th sample; and the argmax function returns the position having the highest value in a 1D array, i.e., in this case it generates the predicted class of the sample.

Fig 3. Inception modules in the GoogLeNet architecture.

(a) The naive inception block that is replaced by (b) the dimension reduction inception block in the GoogLeNet architecture to improve computational efficiency.

Fig 4. Architecture of the GoogLeNet model used in this study.

The inception block is shown in Fig 3(b).

Fig 5. Architecture of the ResNet-18 model used in this study.

Fig 6. Basic architecture of the DenseNet convolutional neural network model.

Fig 7. Confusion matrices obtained on the Kermany pneumonia chest X-ray dataset by the proposed method by 5-fold cross validation.

a) Fold-1. (b) Fold-2. (c) Fold-3. (d) Fold-4. (e) Fold-5.

Fig 8. Confusion matrices obtained on the Radiological Society of North America pneumonia challenge chest X-ray dataset by the proposed method by five-fold cross validation.

a) Fold-1. (b) Fold-2. (c) Fold-3. (d) Fold-4. (e) Fold-5.

Fig 9. Receiver operating characteristic curves obtained by the proposed ensemble method on the two pneumonia chest X-ray datasets used in this research.

(a) Kermany dataset [4]. (b) RSNA challenge dataset [33].

Fig 10. Variation of accuracy rates on the Kermany dataset [4]) achieved by the three base learners, GoogLeNet, ResNet-18, and DenseNet-121 and their ensemble, according to the optimizers chosen for fine tuning.

Fig 11. Variation in performance (accuracy rates) of the ensemble with respect to the number of fixed non-trainable layers in the base learners on the two datasets used in this study.

(a) Kermany dataset [4]. (b) RSNA challenge dataset [33].

Fig 12. Gradient-weighted class activation map (GradCAM) decision visualization of chest X-ray images when the three chosen base learners were used to form the ensemble.

Different regions of the X-rays are the focus of the different models that capture complementary information. Case-1: (a)–(c) show a pneumonic lung X-ray analyzed using the three base learners; the confidence scores of the three base learners are GoogLeNet: 99.99%, ResNet-18: 75.21%, and DenseNet-121: 98.90% Case-2: (d)–(f) show a healthy lung X-ray analyzed using the three base learners; the confidence scores of the three base learners are GoogLeNet: 99.47%, ResNet-18: 97.61%, and DenseNet-121: 98.93%.

Fig 13. Examples of samples from the Kermany dataset where two out of three base learners yielded incorrect predictions, but the ensemble yielded the correct prediction.

Both images are of class “Normal”. (a) Case-1: GoogLeNet predicted “Pneumonia” with a confidence score of 53.1%, ResNet-18 predicted “Pneumonia” with a confidence score of 73.8%, and DenseNet-121 predicted “Normal” with a confidence score of 89.4%. The proposed ensemble framework predicted “Normal” (correct classification) with a confidence rate of 68.1 (b) Case-2: GoogLeNet predicted “Normal” with a confidence score of 98.6%, ResNet-18 predicted “Pneumonia” with a confidence score of 58.3%, and DenseNet-121 predicted “Pneumonia” with a confidence score of 69.3%. The proposed ensemble framework predicted “Normal” (correct classification) with a confidence rate of 66.3%.

Fig 14. Examples of samples from the Kermany dataset [4] that were classified incorrectly by the proposed ensemble framework.

Case-1: (a) shows an image originally belonging to class “Normal” but misclassified as “Pneumonia” by the framework. The GradCAM analysis images are shown in (c), (d), and (e) for GoogLeNet, ResNet-18, and DenseNet-121, respectively. Case-2: (b) shows an image of class “Pneumonia” predicted to belong to the “Normal” class by the framework. The GradCAM analysis images are shown in (f), (g), and (h)for GoogLeNet, ResNet-18, and DenseNet-121, respectively.