Decoding COVID-19 pneumonia: comparison of deep learning and radiomics CT image signatures

Abstract

Purpose: High-dimensional image features that underlie COVID-19 pneumonia remain opaque. We aim to compare feature engineering and deep learning methods to gain insights into the image features that drive CT-based for COVID-19 pneumonia prediction, and uncover CT image features significant for COVID-19 pneumonia from deep learning and radiomics framework.

Methods: A total of 266 patients with COVID-19 and other viral pneumonia with clinical symptoms and CT signs similar to that of COVID-19 during the outbreak were retrospectively collected from three hospitals in China and the USA. All the pneumonia lesions on CT images were manually delineated by four radiologists. One hundred eighty-four patients (n = 93 COVID-19 positive; n = 91 COVID-19 negative; 24,216 pneumonia lesions from 12,001 CT image slices) from two hospitals from China served as discovery cohort for model development. Thirty-two patients (17 COVID-19 positive, 15 COVID-19 negative; 7883 pneumonia lesions from 3799 CT image slices) from a US hospital served as external validation cohort. A bi-directional adversarial network-based framework and PyRadiomics package were used to extract deep learning and radiomics features, respectively. Linear and Lasso classifiers were used to develop models predictive of COVID-19 versus non-COVID-19 viral pneumonia.

Results: 120-dimensional deep learning image features and 120-dimensional radiomics features were extracted. Linear and Lasso classifiers identified 32 high-dimensional deep learning image features and 4 radiomics features associated with COVID-19 pneumonia diagnosis (P < 0.0001). Both models achieved sensitivity > 73% and specificity > 75% on external validation cohort with slight superior performance for radiomics Lasso classifier. Human expert diagnostic performance improved (increase by 16.5% and 11.6% in sensitivity and specificity, respectively) when using a combined deep learning-radiomics model.

Conclusions: We uncover specific deep learning and radiomics features to add insight into interpretability of machine learning algorithms and compare deep learning and radiomics models for COVID-19 pneumonia that might serve to augment human diagnostic performance.

Keywords: AI interpretability; CT chest; Coronavirus disease 2019 pneumonia; Explainable AI; Machine learning.

Fig. 1

Radiomics and artificial intelligence neural network workflow in this study

Fig. 2

Patient enrollment in our study. Asterisk denotes the exposure history defined in our study (for patients from China): history of travel to Wuhan in the last 14 days, history of contact with confirmed COVID-19 patient(s), and history of being in a dense crowd. The relevant exposure history was selected as an inclusion criterion since these patients were high-risk of COVID-19 infection during this period

Fig. 3

The pneumonia lesions on the CT image were used as the input of the BigBiGAN and PyRadiomics. Receiver operating characteristic curves (ROC) and area under curve (AUC) of the linear classifier and Lasso classifier for the differentiation of COVID-19 from other forms of viral pneumonia with clinical symptoms and CT signs similar to those of COVID-19. The four ROC curves in each chart represent the training (red), validation (green), test (blue), and external validation datasets (yellow), respectively

Fig. 4

The CT images of COVID-19 positive (a) and COVID-19 negative (b) with significant different signature values based on the combined feature matrix. Figure a(1) represents a 35-year-old male and CT manifested as bilateral opacities, and linear signature score of 1.32 and Lasso signature score of 0.99; figure a(2) denotes a 43-year-old female and CT manifestation are bilateral ground-glass opacities, vascular thickening, and interlobular septal thickening, with signature scores of 1.23 and 0.99; figure a(3) denotes a 62-year-old male and CT manifestation is bilateral multifocal consolidations. Signature scores are 1.24 and 0.98; figure a(4) represents a 45-year-old female and CT manifested as bilateral peripheral multifocal lesions with signature scores of 1.25 and 0.97; figure b(1) represents a 29-year-old male and CT manifestation is multifocal ground-glass opacities in the left lung. Signature scores are − 0.14 and 0.04; figure b(2) represents a 30-year-old female and CT manifestation is multifocal, mixed ground-glass opacity and consolidation in the right lung. Signature scores are − 0.11 and 0.08; figure b(3) represents a 30-year-old male and CT manifestation is bilateral multifocal consolidation. Signature scores are 0.07 and 0.70; figure b(4) represents a 29-year-old male and CT manifested as mixed densities in the right lung. Signature scores are − 0.17 and 0.03, respectively

Fig. 5

Sensitivity and specificity of the radiologists’ diagnosis on the test datasets without (first round of diagnosis) and with (second round of diagnosis) the assistance of our AI semantic features plus radiomics features