Artificial intelligence-enabled rapid diagnosis of patients with COVID-19

Abstract

For diagnosis of coronavirus disease 2019 (COVID-19), a SARS-CoV-2 virus-specific reverse transcriptase polymerase chain reaction (RT-PCR) test is routinely used. However, this test can take up to 2 d to complete, serial testing may be required to rule out the possibility of false negative results and there is currently a shortage of RT-PCR test kits, underscoring the urgent need for alternative methods for rapid and accurate diagnosis of patients with COVID-19. Chest computed tomography (CT) is a valuable component in the evaluation of patients with suspected SARS-CoV-2 infection. Nevertheless, CT alone may have limited negative predictive value for ruling out SARS-CoV-2 infection, as some patients may have normal radiological findings at early stages of the disease. In this study, we used artificial intelligence (AI) algorithms to integrate chest CT findings with clinical symptoms, exposure history and laboratory testing to rapidly diagnose patients who are positive for COVID-19. Among a total of 905 patients tested by real-time RT-PCR assay and next-generation sequencing RT-PCR, 419 (46.3%) tested positive for SARS-CoV-2. In a test set of 279 patients, the AI system achieved an area under the curve of 0.92 and had equal sensitivity as compared to a senior thoracic radiologist. The AI system also improved the detection of patients who were positive for COVID-19 via RT-PCR who presented with normal CT scans, correctly identifying 17 of 25 (68%) patients, whereas radiologists classified all of these patients as COVID-19 negative. When CT scans and associated clinical history are available, the proposed AI system can help to rapidly diagnose COVID-19 patients.

Extended Data Fig. 1 |. Characteristics of clinical features in the training set, tuning set and test set.

± indicates mean ± standard deviation. * Data in parentheses show the number of COVID-19 positive patients. † Data in parentheses show Interquartile Range (IQR) Other data in parentheses show percentage.

Extended Data Fig. 2 |. Comparisons of the diagnostic performance between AI models and human readers on a test set of 279 cases.

Data were presented in percentage and the 95% confidence interval. The confidence intervals of accuracy were calculated by the exact Clopper-Pearson method. The confidence intervals of the predictive values were calculated by the standard logit confidence intervals.

Extended Data Fig. 3 |. Comparisons of predictions by the joint model and human readers on a test set of 279 cases.

The (+/−) indicates the prediction of the COVID-19 status by the joint model and human readers.

Extended Data Fig. 4 |. ROC curve comparison of the MLP, random forest and SVM models on the tuning set of 92 cases.

Two-sided p-value indicates the significance of difference in performance metric compared with respect to the MLP model by using the DeLong test. The MLP showed no significant difference as compared to the SVM model (p=0.98) and the random forest model (p=0.35). The MLP model was selected in this study due to the highest AUC score.

Fig. 1 |. Illustration of the modeling framework.

Three AI models are used to generate the probability of a patient being COVID-19 (+): the first is based on a chest CT scan, the second on clinical information and the third on a combination of the chest CT scan and clinical information. For evaluation of chest CT scans, each slice was first ranked by the probability of containing a parenchymal abnormality, as predicted by the CNN model (slice selection CNN), which is a pretrained pulmonary tuberculosis (PTB) model that has a 99.4% accuracy to select abnormal lung slices from chest CT scans. The top ten abnormal CT images per patient were put into the second CNN (diagnosis CNN) to predict the likelihood of COVID-19 positivity (P₁). Demographic and clinical data (the patient’s age and sex, exposure history, symptoms and laboratory tests) were put into a machine-learning model to classify COVID-19 positivity (P₂). Features generated by the diagnosis CNN model and the nonimaging clinical information machine-learning model were integrated by an MLP network to generate the final output of the joint model (P₃).

Fig. 2 |. Results of the AI models on the test set.

a, Comparison of the ROC curves for the joint model, the CNN model trained on the basis of CT images, the MLP model trained on the basis of clinical information and two radiologists. b, Comparison of success rates of diagnosing patients who are positive for COVID-19 with normal CT scans. Radiologists were provided with both CT imaging and clinical information in making their diagnoses. c–e, Comparison of the AUCs (c), sensitivities (d) and specificities (e) achieved by the AI models and radiologists. Two-sided P values were calculated by comparing the joint model to the CNN model, the MLP model and the two human readers in sensitivity, specificity and AUC. AUC comparisons were evaluated by the DeLong test; sensitivity and specificity comparisons were calculated by using the exact Clopper–Pearson method to compute the 95% CI shown in parentheses and exact McNemar’s test to calculate the P value.

Fig. 3 |. Examples of chest CT images of patients with COVID-19 and visualization of features correlated to COVID-19 positivity.

For each pair of images, the left image is a CT image showing the segmented lung used as input for the CNN model trained only on CT images and the right image shows the heat map of pixels that the CNN model classified as having SARS-CoV-2 infection (red indicates higher probability). a, A 51-year-old female with fever and history of exposure to SARS-CoV-2. The CNN model identified abnormal features in the right lower lobe (white color), whereas the two radiologists labeled this CT as negative. b, A 52-year-old female who had a history of exposure to SARS-CoV-2 and presented with fever and productive cough. Bilateral peripheral ground-glass opacities (arrows) were labeled by the radiologists and the CNN model predicted positivity based on features in matching areas. c, A 72-year-old female with exposure history to the animal market in Wuhan presented with fever and productive cough. The segmented CT image shows ground-glass opacity in the anterior aspect of the right lung (arrow), whereas the CNN model labeled this CT as negative. d, A 59-year-old female with cough and exposure history. The segmented CT image shows no evidence of pneumonia and the CNN model also labeled this CT as negative.