Evaluation of Feature Selection Techniques for Breast Cancer Risk Prediction

Abstract

This study evaluates several feature ranking techniques together with some classifiers based on machine learning to identify relevant factors regarding the probability of contracting breast cancer and improve the performance of risk prediction models for breast cancer in a healthy population. The dataset with 919 cases and 946 controls comes from the MCC-Spain study and includes only environmental and genetic features. Breast cancer is a major public health problem. Our aim is to analyze which factors in the cancer risk prediction model are the most important for breast cancer prediction. Likewise, quantifying the stability of feature selection methods becomes essential before trying to gain insight into the data. This paper assesses several feature selection algorithms in terms of performance for a set of predictive models. Furthermore, their robustness is quantified to analyze both the similarity between the feature selection rankings and their own stability. The ranking provided by the SVM-RFE approach leads to the best performance in terms of the area under the ROC curve (AUC) metric. Top-47 ranked features obtained with this approach fed to the Logistic Regression classifier achieve an AUC = 0.616. This means an improvement of 5.8% in comparison with the full feature set. Furthermore, the SVM-RFE ranking technique turned out to be highly stable (as well as Random Forest), whereas relief and the wrapper approaches are quite unstable. This study demonstrates that the stability and performance of the model should be studied together as Random Forest and SVM-RFE turned out to be the most stable algorithms, but in terms of model performance SVM-RFE outperforms Random Forest.

Keywords: breast cancer; feature selection; risk prediction model; stability.

Figure 1

Area under the curve using the complete data set without reducing features and different cardinality of the subset of features for different classifiers: AdaBoost.

Figure 2

Area under the curve using the complete data set without reducing features and different cardinality of the subset of features for different classifiers: k-NN.

Figure 3

Area under the curve using the complete data set without reducing features and different cardinality of the subset of features for different classifiers: Logistic Regression.

Figure 4

Area under the curve using the complete data set without reducing features and different cardinality of the subset of features for different classifiers: Multilayer Perceptron.

Figure 5

Area under the curve using the complete data set without reducing features and different cardinality of the subset of features for different classifiers: SVM.

Figure 6

Feature selector stability: (a) Jaccard index for feature subsets with different cardinality; (b) MDS plot of the feature ranking algorithms.

Figure 7

Performance for pre-menopausal data partition (by classifier): (a) BT, (b) k-NN, (c) LR, (d) MLP, (e) SVM.

Figure 8

Performance for postmenopausal data partition (by classifier): (a) BT, (b) k-NN, (c) LR, (d) MLP, (e) SVM.