Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Aug 15:62:101457.
doi: 10.1016/j.nmni.2024.101457. eCollection 2024 Dec.

Automated quantification of SARS-CoV-2 pneumonia with large vision model knowledge adaptation

Zhaohui Liang  1 , Zhiyun Xue  1 , Sivaramakrishnan Rajaraman  1 , Sameer Antani  1
Affiliations

Automated quantification of SARS-CoV-2 pneumonia with large vision model knowledge adaptation

Zhaohui Liang et al. New Microbes New Infect. .

Abstract

Background: Large vision models (LVM) pretrained by large datasets have demonstrated their enormous capacity to understand visual patterns and capture semantic information from images. We proposed a novel method of knowledge domain adaptation with pretrained LVM for a low-cost artificial intelligence (AI) model to quantify the severity of SARS-CoV-2 pneumonia based on frontal chest X-ray (CXR) images.

Methods: Our method used the pretrained LVMs as the primary feature extractor and self-supervised contrastive learning for domain adaptation. An encoder with a 2048-dimensional feature vector output was first trained by self-supervised learning for knowledge domain adaptation. Then a multi-layer perceptron (MLP) was trained for the final severity prediction. A dataset with 2599 CXR images was used for model training and evaluation.

Results: The model based on the pretrained vision transformer (ViT) and self-supervised learning achieved the best performance in cross validation, with mean squared error (MSE) of 23.83 (95 % CI 22.67-25.00) and mean absolute error (MAE) of 3.64 (95 % CI 3.54-3.73). Its prediction correlation has the R 2 of 0.81 (95 % CI 0.79-0.82) and Spearman ρ of 0.80 (95 % CI 0.77-0.81), which are comparable to the current state-of-the-art (SOTA) methods trained by much larger CXR datasets.

Conclusion: The proposed new method has achieved the SOTA performance to quantify the severity of SARS-CoV-2 pneumonia at a significantly lower cost. The method can be extended to other infectious disease detection or quantification to expedite the application of AI in medical research.

Keywords: Artificial intelligence; Large vision model; SARS-CoV2 pneumonia; Self-supervised learning; Vision transformer.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Fig. 1
Fig. 1
CXR image with mRALE annotation.
Fig. 2
Fig. 2
Vision transformer with multi-head attention.
Fig. 3
Fig. 3
Self-supervised learning of LVM for knowledge domain adaptation.
Fig. 4
Fig. 4
LVM training strategies to quantify SARS-CoV-2 pneumonia severity.
Fig. 5
Fig. 5
Distribution of the attentions of LVM.
See this image and copyright information in PMC

References

    1. WHO, WHO Coronavirus (COVID-19) Dashboard, https://covid19.who.int/, last accessed 2024/July/10.
    1. National Institutes of Health. NIH covid-19 treatment guidelines. Clinical spectrum of SARS-CoV-2 Infection. 2020. https://www.covid19treatmentguidelines.nih.gov/overview/clinical-spectrum, last accessed 2024/December/27.
    1. Yang X., Yu Y., Xu J., Shu H., Xia J., Liu H., et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020 May;8(5):475–481. - PMC - PubMed
    1. Wu C., Chen X., Cai Y., Xia J., Zhou X., Xu S., et al. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in wuhan, China. JAMA Intern Med. 2020 Jul 1;180(7):934–943. - PMC - PubMed
    1. Attaway A.H., Scheraga R.G., Bhimraj A., Biehl M., Hatipoglu U. Severe covid-19 pneumonia: pathogenesis and clinical management. BMJ. 2021 Mar 10;372 - PubMed

LinkOut - more resources