Assisting scalable diagnosis automatically via CT images in the combat against COVID-19

Abstract

The pandemic of Coronavirus Disease 2019 (COVID-19) is causing enormous loss of life globally. Prompt case identification is critical. The reference method is the real-time reverse transcription PCR (RT-PCR) assay, whose limitations may curb its prompt large-scale application. COVID-19 manifests with chest computed tomography (CT) abnormalities, some even before the onset of symptoms. We tested the hypothesis that the application of deep learning (DL) to 3D CT images could help identify COVID-19 infections. Using data from 920 COVID-19 and 1,073 non-COVID-19 pneumonia patients, we developed a modified DenseNet-264 model, COVIDNet, to classify CT images to either class. When tested on an independent set of 233 COVID-19 and 289 non-COVID-19 pneumonia patients, COVIDNet achieved an accuracy rate of 94.3% and an area under the curve of 0.98. As of March 23, 2020, the COVIDNet system had been used 11,966 times with a sensitivity of 91.12% and a specificity of 88.50% in six hospitals with PCR confirmation. Application of DL to CT images may improve both efficiency and capacity of case detection and long-term surveillance.

Figure 1

Flow diagram illustrating the division of model development data into training, validation, initial test, and secondary test on the secondary test dataset. The model development cohort consisted of 1197 COVID-19 patients and 1,081 non-COVID-19 patients. The secondary test cohort enrolled 233 COVID-19 patients and 289 non-COVID-19 patients. 285 patients were excluded due to ≥ 2 weeks interval of the first CT scan to the first positive nucleic acid test (244 patients), and severe artifacts (41 patients). The model development cohort was then divided into a training dataset (1316 patients), a validation dataset (305 patients), and an initial test dataset (372 patients) to train and fine-tune COVIDNet. A secondary test was then performed to test the model generalizability by comparing the diagnostic performance of COVIDNet and 8 different expert radiologists on the secondary test dataset. *COVID-19 radiologists: radiologists working at the COVID-19 designated hospitals; §non-COVID-19 radiologists: radiologists not working at the COVID-19 designated hospitals.

Figure 2

t-SNE of CT images of COVID-19 and other causes of pneumonia on the secondary test dataset. G1 depicted chest CT images of COVID-19 pneumonia with the highest differentiation. G2 represented three cases of false-positive prediction of COVIDNet. G3 pointed out nine cases of false-negative prediction of COVIDNet. CT image manifestations of G1 to G3 were illustrated in Supplementary Table 10.

Figure 3

The pipeline of COVIDNet application in the real world.

Figure 4

The performance of COVIDNet in the real-world application. (a) The confusion matrix of COVIDNet in the 6 hospitals, (b–g) the confusion matrix of COVIDNet in each hospital (b–g represents Hospital 1–6, respectively).