Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2021 Dec 1;4(12):e2141096.
doi: 10.1001/jamanetworkopen.2021.41096.

An Artificial Intelligence-Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

Fatemeh Homayounieh  1 , Subba Digumarthy  1 , Shadi Ebrahimian  1 , Johannes Rueckel  2 , Boj Friedrich Hoppe  2 , Bastian Oliver Sabel  2 , Sailesh Conjeti  3 , Karsten Ridder  4 , Markus Sistermanns  4 , Lei Wang  5 , Alexander Preuhs  3 , Florin Ghesu  6 , Awais Mansoor  6 , Mateen Moghbel  1 , Ariel Botwin  1 , Ramandeep Singh  1 , Samuel Cartmell  1 , John Patti  1 , Christian Huemmer  3 , Andreas Fieselmann  3 , Clemens Joerger  3 , Negar Mirshahzadeh  3 , Victorine Muse  1 , Mannudeep Kalra  1
Affiliations
Multicenter Study

An Artificial Intelligence-Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study

Fatemeh Homayounieh et al. JAMA Netw Open. .

Abstract

Importance: Most early lung cancers present as pulmonary nodules on imaging, but these can be easily missed on chest radiographs.

Objective: To assess if a novel artificial intelligence (AI) algorithm can help detect pulmonary nodules on radiographs at different levels of detection difficulty.

Design, setting, and participants: This diagnostic study included 100 posteroanterior chest radiograph images taken between 2000 and 2010 of adult patients from an ambulatory health care center in Germany and a lung image database in the US. Included images were selected to represent nodules with different levels of detection difficulties (from easy to difficult), and comprised both normal and nonnormal control.

Exposures: All images were processed with a novel AI algorithm, the AI Rad Companion Chest X-ray. Two thoracic radiologists established the ground truth and 9 test radiologists from Germany and the US independently reviewed all images in 2 sessions (unaided and AI-aided mode) with at least a 1-month washout period.

Main outcomes and measures: Each test radiologist recorded the presence of 5 findings (pulmonary nodules, atelectasis, consolidation, pneumothorax, and pleural effusion) and their level of confidence for detecting the individual finding on a scale of 1 to 10 (1 representing lowest confidence; 10, highest confidence). The analyzed metrics for nodules included sensitivity, specificity, accuracy, and receiver operating characteristics curve area under the curve (AUC).

Results: Images from 100 patients were included, with a mean (SD) age of 55 (20) years and including 64 men and 36 women. Mean detection accuracy across the 9 radiologists improved by 6.4% (95% CI, 2.3% to 10.6%) with AI-aided interpretation compared with unaided interpretation. Partial AUCs within the effective interval range of 0 to 0.2 false positive rate improved by 5.6% (95% CI, -1.4% to 12.0%) with AI-aided interpretation. Junior radiologists saw greater improvement in sensitivity for nodule detection with AI-aided interpretation as compared with their senior counterparts (12%; 95% CI, 4% to 19% vs 9%; 95% CI, 1% to 17%) while senior radiologists experienced similar improvement in specificity (4%; 95% CI, -2% to 9%) as compared with junior radiologists (4%; 95% CI, -3% to 5%).

Conclusions and relevance: In this diagnostic study, an AI algorithm was associated with improved detection of pulmonary nodules on chest radiographs compared with unaided interpretation for different levels of detection difficulty and for readers with different experience.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Drs Conjeti, Wang, Preuhs, Ghesu, Mansoor, Huemmer, Fieselmann, Joerger, and Mirshahzadeh are full-time paid employees of Siemens Healthineers. Dr Digumarthy reported receiving an honorarium from Siemens Medical Solutions; he reported receiving grant funding from Lunit Inc; he reported consultation fees for independent image analysis from Pfizer, Merck, Novartis, Roche, Bayer, Abbvie, Zai Laboratories, Biengen, Gradalis, and Bristol Mayer Squibb outside the submitted work; he reported receiving grant funding from GE outside the submitted work. Dr Rueckel reported receiving compensation for conference presentations from Siemens Healthineers GmbH outside the submitted work. Dr Sabel reported receiving grants from Siemens Healthcare GmbH Department of Radiology and University Hospital of Ludwig Maximilian University of Munich; he reported receiving compensation for conference presentations from Siemens Healthcare GmbH during the conduct of the study. Dr Sistermanns reported grants from Siemens Healthineers outside the submitted work. Mrs Mirshahzadeh reported receiving personal fees from Siemens Healthcare GmbH during the conduct of the study. Dr Kalra reported receiving grants from Riverain Technologies outside the submitted work. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Flowchart of Chest Radiograph Selection, Inclusion and Exclusion Criteria, Ground Truthing, and Multireader Study
AI indicates artificial intelligence; DICOM, Digital Imaging and Communications in Medicine.
Figure 2.
Figure 2.. Deidentified Radiograph Images From 4 Adult Patients With True Positive Pulmonary Nodules
Panel A, artificial intelligence (AI) helped detect right upper lobe nodule (arrowheads) for 3 junior (J) radiologists as well as improved the confidence score for 1 senior (S) radiologist (score >5 implies positive finding; score ≤5 is negative). Likewise, AI-aided interpretation led to detection of missed right upper lobe nodule (arrowheads) for the radiograph in panel B for 2 junior radiologists and improved confidence of 2 junior radiologists. For panel C, AI helped 2 junior and 2 senior radiologists detect right lower lung nodule (arrowheads) they had missed on unaided interpretation. AI also helped either detect the right mid- and left-lower lung nodules (L1 and L2; arrowheads) (1 junior and 2 senior radiologists for each nodule) or improve confidence for detecting nodules (3 junior and 1 senior radiologists).
See this image and copyright information in PMC

References

    1. Greene R. Francis H. Williams, MD: father of chest radiology in North America. Radiographics. 1991;11(2):325-332. doi:10.1148/radiographics.11.2.2028067 - DOI - PubMed
    1. Yoo H, Lee SH, Arru CD, et al. . AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset. Eur Radiol. 2021;31:9664-9674. doi:10.1007/s00330-021-08074-7 - DOI - PubMed
    1. van Beek EJ, Mirsadraee S, Murchison JT. Lung cancer screening: computed tomography or chest radiographs? World J Radiol. 2015;7(8):189-193. doi:10.4329/wjr.v7.i8.189 - DOI - PMC - PubMed
    1. Del Ciello A, Franchi P, Contegiacomo A, Cicchetti G, Bonomo L, Larici AR. Missed lung cancer: when, where, and why? Diagn Interv Radiol. 2017;23(2):118-126. doi:10.5152/dir.2016.16187 - DOI - PMC - PubMed
    1. Tack D, Howarth N. Missed Lung Lesions: Side-by-Side Comparison of Chest Radiography with MDCT. In: Hodler J, Kubik-Huch RA, von Schulthess GK, eds. Diseases of the Chest, Breast, Heart and Vessels 2019-2022: Diagnostic and Interventional Imaging. Springer; 2019:17-26. doi:10.1007/978-3-030-11149-6_2 - DOI - PubMed

Publication types