The effect of feature normalization methods in radiomics

Abstract

Objectives: In radiomics, different feature normalization methods, such as z-Score or Min-Max, are currently utilized, but their specific impact on the model is unclear. We aimed to measure their effect on the predictive performance and the feature selection.

Methods: We employed fifteen publicly available radiomics datasets to compare seven normalization methods. Using four feature selection and classifier methods, we used cross-validation to measure the area under the curve (AUC) of the resulting models, the agreement of selected features, and the model calibration. In addition, we assessed whether normalization before cross-validation introduces bias.

Results: On average, the difference between the normalization methods was relatively small, with a gain of at most + 0.012 in AUC when comparing the z-Score (mean AUC: 0.707 ± 0.102) to no normalization (mean AUC: 0.719 ± 0.107). However, on some datasets, the difference reached + 0.051. The z-Score performed best, while the tanh transformation showed the worst performance and even decreased the overall predictive performance. While quantile transformation performed, on average, slightly worse than the z-Score, it outperformed all other methods on one out of three datasets. The agreement between the features selected by different normalization methods was only mild, reaching at most 62%. Applying the normalization before cross-validation did not introduce significant bias.

Conclusion: The choice of the feature normalization method influenced the predictive performance but depended strongly on the dataset. It strongly impacted the set of selected features.

Critical relevance statement: Feature normalization plays a crucial role in the preprocessing and influences the predictive performance and the selected features, complicating feature interpretation.

Key points: • The impact of feature normalization methods on radiomic models was measured. • Normalization methods performed similarly on average, but differed more strongly on some datasets. • Different methods led to different sets of selected features, impeding feature interpretation. • Model calibration was not largely affected by the normalization method.

Keywords: Feature normalization; Feature scaling; Feature selection; High-dimensional datasets; Radiomics.

Fig. 1

Flow diagram of the design of the experiments

Fig. 2

Overview of the best-performing models’ predictive performance and model calibration metrics averaged over all repeats. Numbers are reported as mean ± standard deviation

Fig. 3

a Mean rank of the feature normalization methods; mean gain, and maximum gain compared to applying no normalization. b Counts of wins and losses between the normalization methods

Fig. 4

Feature agreement of the best-performing models across repeats. a Agreement of feature selection methods of the best-performing models (in %). b Feature agreement of the selected features of the best-performing models, measured via Intersection-over-Union (in %)

Fig. 5

Differences of the best-performing models when applying feature normalization incorrectly before cross-validation compared to applying it correctly. a Differences averaged over all repeats. b Differences in AUC for each dataset. Numbers are reported as mean ± standard deviation