Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2025 Mar;32(3):1706-1717.
doi: 10.1016/j.acra.2024.11.003. Epub 2024 Nov 25.

Effect of Artificial Intelligence as a Second Reader on the Lung Nodule Detection and Localization Accuracy of Radiologists and Non-radiology Physicians in Chest Radiographs: A Multicenter Reader Study

Dennis Robert  1 , Saigopal Sathyamurthy  2 , Anshul Kumar Singh  2 , Sri Anusha Matta  2 , Manoj Tadepalli  2 , Swetha Tanamala  2 , Vijay Bosemani  3 , Joseph Mammarappallil  4 , Bunty Kundnani  5
Affiliations
Free article
Multicenter Study

Effect of Artificial Intelligence as a Second Reader on the Lung Nodule Detection and Localization Accuracy of Radiologists and Non-radiology Physicians in Chest Radiographs: A Multicenter Reader Study

Dennis Robert et al. Acad Radiol. 2025 Mar.
Free article

Abstract

Rationale and objectives: Missed nodules in chest radiographs (CXRs) are common occurrences. We assessed the effect of artificial intelligence (AI) as a second reader on the accuracy of radiologists and non-radiology physicians in lung nodule detection and localization in CXRs.

Materials and methods: This retrospective study using the multi-reader multi-case design included 300 CXRs acquired from 40 hospitals across the US. All CXRs had a paired follow-up image (chest CT or CXR) to augment the ground truth establishment for the presence and location of nodules on CXRs by five independent thoracic radiologists. 15 readers (nine radiologists and six non-radiology physicians) read each CXR twice in a second-reader paradigm, once without AI and then immediately with AI assistance. The primary analysis assessed the difference in area-under-the-alternative-free-response-receiver-operating-characteristic-curve (AFROC) of readers with and without AI. Case-level area-under-the-receiver-operating-characteristic-curve (AUROC), sensitivity, and specificity were assessed in secondary analyses.

Results: A total of 300 CXRs (147 with nodules, 153 without nodules) from 300 patients (mean age, 64 years ± 15 [standard deviation]; 174 women) were included. The mean AFROC of readers was 0.73 without AI and 0.81 with AI (95% CI of difference, 0.05-0.10). Case-level AUROC was 0.77 without AI and 0.84 with AI (95% CI of difference, 0.04-0.09). Case-level sensitivity was 72.8% and 83.5% (95% CI of difference, 6.8-14.6) and specificity was 71.1% and 72.0% (95% CI of difference, -0.8-2.6) without and with AI, respectively.

Conclusion: Using AI, readers detected and localized more nodules without any significant difference in false positive interpretations.

Keywords: Artificial intelligence; Chest X-ray; Computer-aided detection; Lung nodule; Missed nodule; Multi-reader multi-case study.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Dennis Robert reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Saigopal Sathyamurthy reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Anshul Kumar Singh reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Sri Anusha Matta reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Manoj Tadepalli reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Swetha Tanamala reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Bunty Kundnani reports a relationship with Qure.ai Technologies Private Limited that includes: employment. Vijay Bosemani reports a relationship with Qure.ai Technologies Private Limited that includes: consulting or advisory. Joseph Mammarappallil reports a relationship with Qure.ai Technologies Private Limited that includes: consulting or advisory. Manoj Tadepalli has patent issued to Qure.ai Technologies Private Limited. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

Publication types

LinkOut - more resources