External Test of a Deep Learning Algorithm for Pulmonary Nodule Malignancy Risk Stratification Using European Screening Data

Abstract

Background Low-dose CT screening reduces lung cancer-related deaths but has high rates of false-positive findings. A deep learning (DL) algorithm could improve nodule risk stratification but requires robust external testing. Purpose To externally test a DL algorithm for nodule malignancy risk estimation using pooled data from three large European lung cancer screening trials. Materials and Methods In this retrospective study, a DL algorithm trained on National Lung Screening Trial data was externally tested using baseline CT scans from the Danish Lung Cancer Screening Trial, the Multicentric Italian Lung Detection trial, and the Dutch-Belgian Lung Cancer Screening Trial. Performance was assessed across the pooled cohort and two subsets: subset A, including indeterminate nodules (5-15 mm); and subset B, including cancers size-matched to benign nodules (1:2 ratio). Performance, including the area under the receiver operating characteristic curve (AUC), was compared with the Pan-Canadian Early Detection of Lung Cancer (PanCan) model. Results The pooled cohort included 4146 participants (median age, 58 years; 78% male participants; median smoking history, 38 pack-years) with 7614 benign and 180 malignant nodules. The DL algorithm achieved AUCs of 0.98, 0.96, and 0.94 for cancers diagnosed within 1 year, 2 years, and throughout screening, respectively, compared with 0.98, 0.94, and 0.93 (P = .19, .02, and .46, respectively) for the PanCan model. In subset A (129 malignant and 2086 benign nodules), DL significantly outperformed PanCan across the same cancer diagnosis timeframes (respective AUCs: 0.95, 0.94, and 0.90 vs 0.91, 0.88, and 0.86; all P < .05). At 100% sensitivity for cancers diagnosed within 1 year, DL classified 68.1% of benign cases as low risk versus 47.4% for the PanCan model, a 39.4% relative reduction in false-positive findings. In subset B (180 malignant and 360 benign nodules), the AUC of the DL algorithm versus the PanCan model was 0.79 versus 0.60 (P < .01), respectively. Conclusion The DL algorithm outperformed the PanCan model across multiple European screening datasets, demonstrating superior malignancy prediction while substantially reducing false-positive classifications for indeterminate nodules. © RSNA, 2025 Supplemental material is available for this article.