Radiomics analysis to predict pulmonary nodule malignancy using machine learning approaches

Abstract

Background: Low-dose CT screening can reduce lung cancer-related mortality. However, most screen-detected pulmonary abnormalities do not develop into cancer and it often remains challenging to identify malignant nodules, particularly among indeterminate nodules. We aimed to develop and assess prediction models based on radiological features to discriminate between benign and malignant pulmonary lesions detected on a baseline screen.

Methods: Using four international lung cancer screening studies, we extracted 2060 radiomic features for each of 16 797 nodules (513 malignant) among 6865 participants. After filtering out low-quality radiomic features, 642 radiomic and 9 epidemiological features remained for model development. We used cross-validation and grid search to assess three machine learning (ML) models (eXtreme Gradient Boosted Trees, random forest, least absolute shrinkage and selection operator (LASSO)) for their ability to accurately predict risk of malignancy for pulmonary nodules. We report model performance based on the area under the curve (AUC) and calibration metrics in the held-out test set.

Results: The LASSO model yielded the best predictive performance in cross-validation and was fit in the full training set based on optimised hyperparameters. Our radiomics model had a test-set AUC of 0.93 (95% CI 0.90 to 0.96) and outperformed the established Pan-Canadian Early Detection of Lung Cancer model (AUC 0.87, 95% CI 0.85 to 0.89) for nodule assessment. Our model performed well among both solid (AUC 0.93, 95% CI 0.89 to 0.97) and subsolid nodules (AUC 0.91, 95% CI 0.85 to 0.95).

Conclusions: We developed highly accurate ML models based on radiomic and epidemiological features from four international lung cancer screening studies that may be suitable for assessing indeterminate screen-detected pulmonary nodules for risk of malignancy.

Keywords: clinical epidemiology; imaging/CT MRI etc; lung cancer.

Figure 1.

Schematic for the analytic framework used in this study. Data were partitioned into training/validation and testing splits using group-based random sampling to ensure all nodules for a participant were in a single set to avoid data leakage. Radiomic features were extracted and subject to filtering to exclude low-quality and highly-redundant features. K-fold cross-validation was performed to identify the optimal machine learning (ML) model and the optimal set of hyperparameters. The final ML model was fitted to the entire training data set and tested for out-of-sample performance in the hold-out test data; discrimination and calibration performance metrics are reported.

Figure 2.

Receiver operating characteristic (ROC) curves for our INTEGRAL-Radiomics models and the established PanCan Model. Area under the curve (AUC) and 95% confidence intervals are reported.

Figure 3.

Calibration of our INTEGRAL-Radiomics model and the PanCan model (McWilliams et al., 2013) in hold-out test-data. (A) Model-predicted risks versus observed risks across quintiles of model-predicted risks. The diagonal dashed line indicates perfect calibration. (B) Observed and expected (model-predicted) number of malignant pulmonary nodules (per 1,000 nodules) including the calibration ratio (Exp / Obs) and difference (Exp - Obs). Calibration ratios less than 1 (or differences less than 0) indicate underestimation of risk and ratios greater than 1 (or differences greater than 0) indicate overestimation of risk.