Deep-learning algorithms for the interpretation of chest radiographs to aid in the triage of COVID-19 patients: A multicenter retrospective study

Abstract

The recent medical applications of deep-learning (DL) algorithms have demonstrated their clinical efficacy in improving speed and accuracy of image interpretation. If the DL algorithm achieves a performance equivalent to that achieved by physicians in chest radiography (CR) diagnoses with Coronavirus disease 2019 (COVID-19) pneumonia, the automatic interpretation of the CR with DL algorithms can significantly reduce the burden on clinicians and radiologists in sudden surges of suspected COVID-19 patients. The aim of this study was to evaluate the efficacy of the DL algorithm for detecting COVID-19 pneumonia on CR compared with formal radiology reports. This is a retrospective study of adult patients that were diagnosed as positive COVID-19 cases based on the reverse transcription polymerase chain reaction among all the patients who were admitted to five emergency departments and one community treatment center in Korea from February 18, 2020 to May 1, 2020. The CR images were evaluated with a publicly available DL algorithm. For reference, CR images without chest computed tomography (CT) scans classified as positive for COVID-19 pneumonia were used given that the radiologist identified ground-glass opacity, consolidation, or other infiltration in retrospectively reviewed CR images. Patients with evidence of pneumonia on chest CT scans were also classified as COVID-19 pneumonia positive outcomes. The overall sensitivity and specificity of the DL algorithm for detecting COVID-19 pneumonia on CR were 95.6%, and 88.7%, respectively. The area under the curve value of the DL algorithm for the detection of COVID-19 with pneumonia was 0.921. The DL algorithm demonstrated a satisfactory diagnostic performance comparable with that of formal radiology reports in the CR-based diagnosis of pneumonia in COVID-19 patients. The DL algorithm may offer fast and reliable examinations that can facilitate patient screening and isolation decisions, which can reduce the medical staff workload during COVID-19 pandemic situations.

Fig 1. Flow chart of reference standard for COVID-19 with pneumonia and COVID-19 without pneumonia.

CR, chest radiography; CT, computed tomography; RT-PCR, reverse transcription polymerase chain reaction.

Fig 2. Results of the deep learning (DL) algorithm analysis of the localization of COVID-19 with pneumonia.

(A, B, C) Chest radiography (D, E, F) DL algorithm heatmap overlaid on the image feature related to pneumonia. (G, H) Computed tomography scan, showing lung consolidation and ground-glass opacities that are suitably localized and detected by the DL algorithm.

Fig 3. False-positive interpretations of DL algorithm.

DL algorithm heatmap overlaid on image feature related to (A) interstitial thickening and (B, C) normal vascular marking.

Fig 4. False-negative interpretations of DL algorithm.

(A, B) DL algorithm classified chest radiography as negative for pneumonia. Chest computed tomography scans show (C) focal GGOs in the right middle lobe, (D) multifocal consolidations in the right lower lobe, and (E, F) small amounts of GGOs.