Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov 25;8(6):2815-2827.
doi: 10.3390/tomography8060235.

Diagnostic Performance in Differentiating COVID-19 from Other Viral Pneumonias on CT Imaging: Multi-Reader Analysis Compared with an Artificial Intelligence-Based Model

Francesco Rizzetto  1   2 , Luca Berta  3 , Giulia Zorzi  3   4 , Antonino Cincotta  1   2 , Francesca Travaglini  1 , Diana Artioli  1 , Silvia Nerini Molteni  5 , Chiara Vismara  5 , Francesco Scaglione  5   6 , Alberto Torresin  3   7 , Paola Enrica Colombo  3   7 , Luca Alessandro Carbonaro  1   6 , Angelo Vanzulli  1   6
Affiliations

Diagnostic Performance in Differentiating COVID-19 from Other Viral Pneumonias on CT Imaging: Multi-Reader Analysis Compared with an Artificial Intelligence-Based Model

Francesco Rizzetto et al. Tomography. .

Abstract

Growing evidence suggests that artificial intelligence tools could help radiologists in differentiating COVID-19 pneumonia from other types of viral (non-COVID-19) pneumonia. To test this hypothesis, an R-AI classifier capable of discriminating between COVID-19 and non-COVID-19 pneumonia was developed using CT chest scans of 1031 patients with positive swab for SARS-CoV-2 (n = 647) and other respiratory viruses (n = 384). The model was trained with 811 CT scans, while 220 CT scans (n = 151 COVID-19; n = 69 non-COVID-19) were used for independent validation. Four readers were enrolled to blindly evaluate the validation dataset using the CO-RADS score. A pandemic-like high suspicion scenario (CO-RADS 3 considered as COVID-19) and a low suspicion scenario (CO-RADS 3 considered as non-COVID-19) were simulated. Inter-reader agreement and performance metrics were calculated for human readers and R-AI classifier. The readers showed good agreement in assigning CO-RADS score (Gwet's AC2 = 0.71, p < 0.001). Considering human performance, accuracy = 78% and accuracy = 74% were obtained in the high and low suspicion scenarios, respectively, while the AI classifier achieved accuracy = 79% in distinguishing COVID-19 from non-COVID-19 pneumonia on the independent validation dataset. The R-AI classifier performance was equivalent or superior to human readers in all comparisons. Therefore, a R-AI classifier may support human readers in the difficult task of distinguishing COVID-19 from other types of viral pneumonia on CT imaging.

Keywords: COVID-19; artificial intelligence; lung; radiomics; tomography (X-ray computed).

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Exemplary cases of different situations occurred in the classification task. For each case, the probability output of the R-AI classifier and the CO-RADS scores assigned by the readers are reported. Final diagnoses by molecular testing were: (a) parainfluenza virus 4 pneumonia; (b) COVID-19 pneumonia; (c) rhinovirus pneumonia; (d) COVID-19 pneumonia.
Figure 2
Figure 2
Confusion matrices of the global performance of the four readers in the high and low suspicion scenarios and the radiomic-based artificial intelligence (R-AI) classifier in distinguishing COVID-19 from other types of viral pneumonia.
Figure 3
Figure 3
Comparison between the accuracy of the radiomic-based artificial intelligence (R-AI) classifier and the four readers in differentiating COVID-19 from other types of viral pneumonia in both high and low suspicion scenarios. Bars on graph boxes represent the 95% confidence interval of the accuracy values. Significant results of pairwise McNemar test after Bonferroni’s correction were reported.
Figure 4
Figure 4
Comparison between the performance of the radiomic-based artificial intelligence (R-AI) classifier and the four readers in differentiating COVID-19 from other types of viral pneumonia in the subset (n = 43) of patients who were assigned a CO-RADS 3 score by two or more. Bars on graph boxes represent the 95% confidence interval of the accuracy values. Significant results of pairwise McNemar test after Bonferroni’s correction were reported.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Johns Hopkins University . COVID-19 Map. Johns Hopkins Coronavirus Resource Center; Singapore: 2021. [(accessed on 20 August 2022)]. Available online: https://coronavirus.jhu.edu/map.html.
    1. Rizzetto F., Perillo N., Artioli D., Travaglini F., Cuccia A., Zannoni S., Tombini V., Di Domenico S.L., Albertini V., Bergamaschi M., et al. Correlation between lung ultrasound and chest CT patterns with estimation of pulmonary burden in COVID-19 patients. Eur. J. Radiol. 2021;138:109650. doi: 10.1016/j.ejrad.2021.109650. - DOI - PMC - PubMed
    1. Yang W., Sirajuddin A., Zhang X., Liu G., Teng Z., Zhao S., Lu M. The role of imaging in 2019 novel coronavirus pneumonia (COVID-19) Eur. Radiol. 2020;30:4874–4882. doi: 10.1007/s00330-020-06827-4. - DOI - PMC - PubMed
    1. Parekh M., Donuru A., Balasubramanya R., Kapur S. Review of the Chest CT Differential Diagnosis of Ground-Glass Opacities in the COVID Era. Radiology. 2020;297:E289–E302. doi: 10.1148/radiol.2020202504. - DOI - PMC - PubMed
    1. Garrana S.H., Som A., Ndakwah G.S., Yeung T., Febbo J., Heeger A.P., Lang M., McDermott S., Mendoza D.P., Zhang E.W., et al. Comparison of Chest CT Findings of COVID-19, Influenza, and Organizing Pneumonia: A Multireader Study. Am. J. Roentgenol. 2021;217:1093–1102. doi: 10.2214/AJR.21.25640. - DOI - PubMed