Radiomic 'Stress Test': exploration of a deep learning radiomic model in a high-risk prospective lung nodule cohort

Abstract

Background: Indeterminate pulmonary nodules (IPNs) are commonly biopsied to ascertain a diagnosis of lung cancer, but many are ultimately benign. The Lung Cancer Prediction (LCP) score is a commercially available deep learning radiomic model with strong diagnostic performance in incidentally identified IPNs, but its potential use to reduce the need for invasive procedures has not been evaluated in patients with nodules for which a biopsy has been recommended.

Methods: In this prospectively collected, retrospective blinded evaluation, the probability of cancer in consecutively biopsied IPNs at a tertiary care centre was calculated using the Mayo Clinic prediction model and categorised into low, intermediate and high-probability groups by applying <10% no-test and >70% treatment thresholds per British Thoracic Society guidelines. We evaluated the diagnostic performance of the Mayo Clinic model, the LCP radiomic model and an integrated model combining the LCP score with statistically selected clinical variables (age, spiculation and upper lobe location) using stepwise logistic regression. Performance was assessed using area under the receiver operating characteristic curve (AUC), F1 score and reclassification analysis based on the bias-corrected clinical net reclassification index.

Results: The study population included 196 malignant and 125 benign IPNs (61% prevalence of malignancy). The Mayo Clinic model's AUC was 0.69 (0.63-0.75), LCP's AUC was 0.67 (0.61-0.73) and the integrated model combining LCP with statistically selected clinical variables (age, spiculation and upper lobe location) had the highest AUC of 0.75 (0.69-0.80). The integrated model demonstrated improved classification, with an F1 score of 0.645 (0.572-0.716) and a significantly higher AUC compared with the Mayo Clinic model (p=0.046). Reclassification analysis showed a clinical net reclassification index of 0.36 (0.21-0.53) for benign IPNs with eight correctly downgraded intermediate-risk benign nodules and no malignant nodules misclassified into the low-risk category.

Conclusion: Incorporating LCP with select clinical variables results in an improvement in malignancy risk prediction and nodule classification and could reduce unnecessary invasive biopsies for IPNs.

Keywords: Clinical Epidemiology; Lung Cancer; Non-Small Cell Lung Cancer.

Figure 1. Receiver operating characteristic (ROC) curves for all models. AUC, area under the receiver operating characteristic curve; LCP, Lung Cancer Prediction score; Mayo, Mayo model; Mayo Select, Mayo model excluding all radiographic variables; ROC, receiver operating characteristic.