Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 19;14(1):1672.
doi: 10.1038/s41598-024-52135-y.

"KAIZEN" method realizing implementation of deep-learning models for COVID-19 CT diagnosis in real world hospitals

Affiliations

"KAIZEN" method realizing implementation of deep-learning models for COVID-19 CT diagnosis in real world hospitals

Naoki Okada et al. Sci Rep. .

Abstract

Numerous COVID-19 diagnostic imaging Artificial Intelligence (AI) studies exist. However, none of their models were of potential clinical use, primarily owing to methodological defects and the lack of implementation considerations for inference. In this study, all development processes of the deep-learning models are performed based on strict criteria of the "KAIZEN checklist", which is proposed based on previous AI development guidelines to overcome the deficiencies mentioned above. We develop and evaluate two binary-classification deep-learning models to triage COVID-19: a slice model examining a Computed Tomography (CT) slice to find COVID-19 lesions; a series model examining a series of CT images to find an infected patient. We collected 2,400,200 CT slices from twelve emergency centers in Japan. Area Under Curve (AUC) and accuracy were calculated for classification performance. The inference time of the system that includes these two models were measured. For validation data, the slice and series models recognized COVID-19 with AUCs and accuracies of 0.989 and 0.982, 95.9% and 93.0% respectively. For test data, the models' AUCs and accuracies were 0.958 and 0.953, 90.0% and 91.4% respectively. The average inference time per case was 2.83 s. Our deep-learning system realizes accuracy and inference speed high enough for practical use. The systems have already been implemented in four hospitals and eight are under progression. We released an application software and implementation code for free in a highly usable state to allow its use in Japan and globally.

PubMed Disclaimer

Conflict of interest statement

Naoki Okada received grants from the Japanese Cabinet Secretariat (438-2020-5E, 834-2021-4A, 847-2022-2C, 847-2022-2D). Naoki Okada and Shusuke Inoue are board members of fcuro inc. Shun Honda and Yuichiro Hirano are employees of fcuro inc. Other authors declare no competing interests.

Figures

Figure 1
Figure 1
Visual abstract. The overview of our work is described in this figure. A large number of CT images were collected and labeled by eight radiologists. Two binary classification models were trained and evaluated by these image datasets. An inference program to execute these models was constructed and implemented to real-world hospitals. All of the process was conducted based on the “KAIZEN checklist”.
Figure 2
Figure 2
Flowchart of the process for inclusion and exclusion of the collected patients’ data. After exclusion, only CT images at the initial imaging of each patient are included in the slice model. All CT series, including follow-up, are included in the series model.
Figure 3
Figure 3
AI system architecture. Raw DICOM images are standardized and molded in the pre-processing unit. These images are input into each of the models in the diagnostic model unit to output probability scores for COVID-19.
Figure 4
Figure 4
ROC curves of the slice and series models. The ROC curves of the slice and series models for the validation and test data are shown in Fig. 3. The AUC values and their 95% confidence intervals are also shown.
Figure 5
Figure 5
Flowchart of our inference process. Dashed arrows indicate the use of outputs in the past steps.
Figure 6
Figure 6
Hospitals that implement our AI system.

Similar articles

Cited by

References

    1. Binnicker MJ. Challenges and controversies to testing for COVID-19. J. Clin. Microbiol. 2020 doi: 10.1128/JCM.01695-20. - DOI - PMC - PubMed
    1. World Health Organization. Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases: Interim guidance (2020) https://apps.who.int/iris/bitstream/handle/10665/331329/WHO-COVID-19-lab....
    1. Corman VM, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020 doi: 10.2807/1560-7917.ES.2020.25.3.2000045. - DOI - PMC - PubMed
    1. Kanne JP, Little BP, Chung JH, Elicker BM, Ketai LH. Essentials for radiologists on COVID-19: An update—Radiology scientific expert panel. Radiology. 2020;296:E113–E114. doi: 10.1148/radiol.2020200527. - DOI - PMC - PubMed
    1. Shi H, et al. Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: A descriptive study. Lancet Infect. Dis. 2020;20:425–434. doi: 10.1016/S1473-3099(20)30086-4. - DOI - PMC - PubMed