Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2022 Aug 21;46(10):62.
doi: 10.1007/s10916-022-01850-y.

Multicenter Study on COVID-19 Lung Computed Tomography Segmentation with varying Glass Ground Opacities using Unseen Deep Learning Artificial Intelligence Paradigms: COVLIAS 1.0 Validation

Jasjit S Suri  1   2 , Sushant Agarwal  3   4 , Luca Saba  5 , Gian Luca Chabert  5 , Alessandro Carriero  6 , Alessio Paschè  5 , Pietro Danna  5 , Armin Mehmedović  7 , Gavino Faa  8 , Tanay Jujaray  3   9 , Inder M Singh  10 , Narendra N Khanna  11 , John R Laird  12 , Petros P Sfikakis  13 , Vikas Agarwal  14 , Jagjit S Teji  15 , Rajanikant R Yadav  16 , Ferenc Nagy  17 , Zsigmond Tamás Kincses  18 , Zoltan Ruzsa  19 , Klaudija Viskovic  7 , Mannudeep K Kalra  20
Affiliations
Multicenter Study

Multicenter Study on COVID-19 Lung Computed Tomography Segmentation with varying Glass Ground Opacities using Unseen Deep Learning Artificial Intelligence Paradigms: COVLIAS 1.0 Validation

Jasjit S Suri et al. J Med Syst. .

Abstract

Variations in COVID-19 lesions such as glass ground opacities (GGO), consolidations, and crazy paving can compromise the ability of solo-deep learning (SDL) or hybrid-deep learning (HDL) artificial intelligence (AI) models in predicting automated COVID-19 lung segmentation in Computed Tomography (CT) from unseen data leading to poor clinical manifestations. As the first study of its kind, "COVLIAS 1.0-Unseen" proves two hypotheses, (i) contrast adjustment is vital for AI, and (ii) HDL is superior to SDL. In a multicenter study, 10,000 CT slices were collected from 72 Italian (ITA) patients with low-GGO, and 80 Croatian (CRO) patients with high-GGO. Hounsfield Units (HU) were automatically adjusted to train the AI models and predict from test data, leading to four combinations-two Unseen sets: (i) train-CRO:test-ITA, (ii) train-ITA:test-CRO, and two Seen sets: (iii) train-CRO:test-CRO, (iv) train-ITA:test-ITA. COVILAS used three SDL models: PSPNet, SegNet, UNet and six HDL models: VGG-PSPNet, VGG-SegNet, VGG-UNet, ResNet-PSPNet, ResNet-SegNet, and ResNet-UNet. Two trained, blinded senior radiologists conducted ground truth annotations. Five types of performance metrics were used to validate COVLIAS 1.0-Unseen which was further benchmarked against MedSeg, an open-source web-based system. After HU adjustment for DS and JI, HDL (Unseen AI) > SDL (Unseen AI) by 4% and 5%, respectively. For CC, HDL (Unseen AI) > SDL (Unseen AI) by 6%. The COVLIAS-MedSeg difference was < 5%, meeting regulatory guidelines.Unseen AI was successfully demonstrated using automated HU adjustment. HDL was found to be superior to SDL.

Keywords: And AI; COVID-19; Glass ground opacities; Hounsfield units; Hybrid deep learning; Lung CT; Segmentation; Solo deep learning.

PubMed Disclaimer

Conflict of interest statement

There are no Conflict of Interest.

Figures

Fig. 1
Fig. 1
Overlay of segmentation results (red) from the ResNet-SegNet HDL models trained without adjusting the HU level. The white arrow represents the region where the ResNet-SegNet HDL model under-estimates the lung region
Fig. 2
Fig. 2
Sample CT scans taken from raw CRO data sets
Fig. 3
Fig. 3
Sample CT scans taken from raw ITA data sets
Fig. 4
Fig. 4
VGG-PSPNet architecture
Fig. 5
Fig. 5
ResNet-PSPNet architecture
Fig. 6
Fig. 6
VGG-UNet architecture
Fig. 7
Fig. 7
ResNet-UNet architecture
Fig. 8
Fig. 8
Accuracy and loss plot for the nine AI models for the training on the CRO dataset
Fig. 9
Fig. 9
Accuracy and loss plot for the nine AI models for the training on the ITA dataset
Fig. 10
Fig. 10
Visual overlays (set 1) showing the AI (green) output against the GT (red) for Seen analysis
Fig. 11
Fig. 11
Visual overlays (set 2) showing the AI (green) output against the GT (red) for Seen analysis
Fig. 12
Fig. 12
Visual overlays (set 1) showing the AI (green) output against the GT (red) for Unseen analysis
Fig. 13
Fig. 13
Visual overlays (set 2) showing the AI (green) output against the GT (red) for Unseen analysis
Fig. 14
Fig. 14
Cumulative frequency plot for Dice using Seen analysis
Fig. 15
Fig. 15
Cumulative frequency plot for Dice using Unseen analysis
Fig. 16
Fig. 16
Cumulative frequency plot for Jaccard using Seen analysis
Fig. 17
Fig. 17
Cumulative frequency plot for Jaccard using Unseen analysis
Fig. 18
Fig. 18
BA plot for Seen analysis
Fig. 19
Fig. 19
BA plot for Unseen analysis
Fig. 20
Fig. 20
CC plot for Seen analysis
Fig. 21
Fig. 21
CC plot for Unseen analysis
Fig. 22
Fig. 22
Cumulative frequency plot of DS for MedSeg for ITA (left) and CRO (right) data sets
Fig. 23
Fig. 23
Cumulative frequency plot of JI for MedSeg for ITA data (left) and CRO data (right)
Fig. 24
Fig. 24
CC plot for MedSeg vs. GT for ITA (left) and CRO (right)
Fig. 25
Fig. 25
BA plot for MedSeg vs. GT for ITA (left) and CRO (right)
Fig. 26
Fig. 26
Overlay of segmentation results from the ResNet-SegNet model trained without adjusting the HU level (red) and after adjusting the HU level (green). The white arrow represents the under-estimated region and the red arrows represent the same region estimated accurately by the ResNet-SegNet model
Fig. 27
Fig. 27
Left: Number of NN layers. Right: Size of the final AI models used in COVLIAS 1.0
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. WHO Coronavirus (COVID-19) Dashboard. Available online: https://covid19.who.int/ (accessed on 24 Jan 2022).
    1. Cucinotta D, Vanelli M. WHO Declares COVID-19 a Pandemic. Acta Biomed. 2020;91:157–160. doi: 10.23750/abm.v91i1.9397. - DOI - PMC - PubMed
    1. Saba L, Gerosa C, Fanni D, Marongiu F, La Nasa G, Caocci G, Barcellona D, Balestrieri A, Coghe F, Orru G, et al. Molecular pathways triggered by COVID-19 in different organs: ACE2 receptor-expressing cells under attack? A review. Eur Rev Med Pharmacol Sci. 2020;24:12609–12622. doi: 10.26355/eurrev_202012_24058. - DOI - PubMed
    1. Suri JS, Puvvula A, Biswas M, Majhail M, Saba L, Faa G, Singh IM, Oberleitner R, Turk M, Chadha PS, et al. COVID-19 pathways for brain and heart injury in comorbidity patients: A role of medical imaging and artificial intelligence-based COVID severity classification: A review. Comput Biol Med. 2020;124:103960. doi: 10.1016/j.compbiomed.2020.103960. - DOI - PMC - PubMed
    1. Cau R, Bassareo PP, Mannelli L, Suri JS, Saba L. Imaging in COVID-19-related myocardial injury. Int J Cardiovasc Imaging. 2021;37:1349–1360. doi: 10.1007/s10554-020-02089-9. - DOI - PMC - PubMed

Publication types