Deep learning in chest radiography: Detection of findings and presence of change

Abstract

Background: Deep learning (DL) based solutions have been proposed for interpretation of several imaging modalities including radiography, CT, and MR. For chest radiographs, DL algorithms have found success in the evaluation of abnormalities such as lung nodules, pulmonary tuberculosis, cystic fibrosis, pneumoconiosis, and location of peripherally inserted central catheters. Chest radiography represents the most commonly performed radiological test for a multitude of non-emergent and emergent clinical indications. This study aims to assess accuracy of deep learning (DL) algorithm for detection of abnormalities on routine frontal chest radiographs (CXR), and assessment of stability or change in findings over serial radiographs.

Methods and findings: We processed 874 de-identified frontal CXR from 724 adult patients (> 18 years) with DL (Qure AI). Scores and prediction statistics from DL were generated and recorded for the presence of pulmonary opacities, pleural effusions, hilar prominence, and enlarged cardiac silhouette. To establish a standard of reference (SOR), two thoracic radiologists assessed all CXR for these abnormalities. Four other radiologists (test radiologists), unaware of SOR and DL findings, independently assessed the presence of radiographic abnormalities. A total 724 radiographs were assessed for detection of findings. A subset of 150 radiographs with follow up examinations was used to asses change over time. Data were analyzed with receiver operating characteristics analyses and post-hoc power analysis.

Results: About 42% (305/ 724) CXR had no findings according to SOR; single and multiple abnormalities were seen in 23% (168/724) and 35% (251/724) of CXR. There was no statistical difference between DL and SOR for all abnormalities (p = 0.2-0.8). The area under the curve (AUC) for DL and test radiologists ranged between 0.837-0.929 and 0.693-0.923, respectively. DL had lowest AUC (0.758) for assessing changes in pulmonary opacities over follow up CXR. Presence of chest wall implanted devices negatively affected the accuracy of DL algorithm for evaluation of pulmonary and hilar abnormalities.

Conclusions: DL algorithm can aid in interpretation of CXR findings and their stability over follow up CXR. However, in its present version, it is unlikely to replace radiologists due to its limited specificity for categorizing specific findings.

Fig 1. Flow-chart diagram illustrating development, validation, and testing of DL algorithm.

Fig 2. ROC curves for the DL algorithm and the four test radiologists (R1, R2, R3 and R4) in for pulmonary opacities (O), pleural effusion (PE), hilar prominence (HP) and enlarged cardiac silhouette (C).

Fig 3. True positive pulmonary opacities.

Frontal CXR belonging to two separate patients. Unprocessed CXR (a, c) demonstrate nodular opacities in the right lung. The corresponding heat maps from DL algorithm (b, d) accurately detected and annotated (in red) these abnormalities.

Fig 4

Frontal chest radiograph (a) without radiographic abnormality. The DL algorithm generated heat map (b) labels false positive pleural effusion on the right side.

Fig 5

Implanted Port catheter projects in the left mid lung zone (a). Heat map from DL algorithm misinterpreted the port catheter as a focal pulmonary opacity. This is apparent on the accompanying heat map image (b).

Fig 6

Frontal CXR demonstrates a subtle patchy opacity in right mid zone (a), which is annotated (in red) on the heat map generated from DL algorithm (b). On subsequent follow-up CXR (c), the heat map did not mark any abnormality (resolution).