Evaluation of the Reliability and the Performance of Magnetic Resonance Imaging Radiomics in the Presence of Randomly Generated Irrelevant Features for Prostate Cancer

Abstract

Radiomics has the potential to aid prostate cancer (PC) diagnoses and prediction by analyzing and modeling quantitative features extracted from clinical imaging. However, its reliability has been a concern, possibly due to its high-dimensional nature. This study aims to quantitatively investigate the impact of randomly generated irrelevant features on MRI radiomics feature selection, modeling, and performance by progressively adding randomly generated features. Two multiparametric-MRI radiomics PC datasets were used (dataset 1 (n = 260), dataset 2 (n = 100)). The endpoint was to differentiate pathology-confirmed clinically significant (Gleason score (GS) ≥ 7) from insignificant (GS < 7) PC. Random features were generated at 12 levels with a 10% increment from 0% to 100% and an additional 5%. Three feature selection algorithms and two classifiers were used to build the models. The area under the curve and accuracy were used to evaluate the model's performance. Feature importance was calculated to assess features' contributions to the models. The metrics of each model were compared using an ANOVA test with a Bonferroni correction. A slight tendency to select more random features with the increasing number of random features introduced to the datasets was observed. However, the performance of the radiomics-built models was not significantly affected, which was partially due to the higher contribution of radiomics features toward the models compared to the random features. These reliability effects also vary among datasets. In conclusion, while the inclusion of additional random features may still slightly impact the performance of the feature selection, it may not have a substantial impact on the MRI radiomics model performance.

Keywords: MRI; machine learning; prostate cancer; radiomics; random features; reliability.

Figure 1

The ratio of the random features selected by the feature selection (least absolute shrinkage and selection operator (LASSO), minimum redundancy maximum relevance (MRMR), and recursive feature elimination (RFE)) with different maximum thresholds across different levels of additional random features.

Figure 2

The Jaccard Similarity Coefficient (JSC) calculation comparing the selected radiomics features in all 20 repetitions by every feature selection method (least absolute shrinkage and selection operator (LASSO), minimum redundancy maximum relevance (MRMR), and recursive feature elimination (RFE)) across different levels of additional random features for the dataset.

Figure 3

The area under the curve (AUC) performance of the models with different combinations of feature selection (least absolute shrinkage and selection operator (LASSO), minimum redundancy maximum relevance (MRMR), recursive feature elimination (RFE)), and classifiers (LASSO and random forest (RF)).

Figure 4

The accuracy of the models with different combinations of feature selection (least absolute shrinkage and selection operator (LASSO), minimum redundancy maximum relevance (MRMR), recursive feature elimination (RFE)), and classifiers (LASSO and random forest (RF)).

Figure 5

The importance of the top 10 most important radiomics and random features toward the radiomics model along with the total mean contribution of random features using random forest (RF) and least absolute shrinkage and selection operator (LASSO) classifiers for dataset 1 (a,b), respectively, and dataset 2 (c,d), respectively, with the maximum threshold set to 20 for the feature selection methods across every repetition and different feature selection methods for the different ratios of additional random features.