Enhancing stroke disease classification through machine learning models via a novel voting system by feature selection techniques

Abstract

Heart disease remains a leading cause of mortality and morbidity worldwide, necessitating the development of accurate and reliable predictive models to facilitate early detection and intervention. While state of the art work has focused on various machine learning approaches for predicting heart disease, but they could not able to achieve remarkable accuracy. In response to this need, we applied nine machine learning algorithms XGBoost, logistic regression, decision tree, random forest, k-nearest neighbors (KNN), support vector machine (SVM), gaussian naïve bayes (NB gaussian), adaptive boosting, and linear regression to predict heart disease based on a range of physiological indicators. Our approach involved feature selection techniques to identify the most relevant predictors, aimed at refining the models to enhance both performance and interpretability. The models were trained, incorporating processes such as grid search hyperparameter tuning, and cross-validation to minimize overfitting. Additionally, we have developed a novel voting system with feature selection techniques to advance heart disease classification. Furthermore, we have evaluated the models using key performance metrics including accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (ROC AUC). Among the models, XGBoost demonstrated exceptional performance, achieving 99% accuracy, precision, F1-Score, 98% recall, and 100% ROC AUC. This study offers a promising approach to early heart disease diagnosis and preventive healthcare.

Fig 1. System architecture of this study.

Fig 2. Histogram of attributes.

Fig 3. Feature selection procedure.

Fig 4. Illustration of all features correlation.

Fig 5. Graphical representation of all models confusion matrix.

Fig 6. The ROC curve for the experiment.

Fig 7

(a) XGBoost Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (b) RF Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (c) KNN Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (d) SVM Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (e) ABR Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (f) NB-Gaussian Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (g) LR Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (h) Linear Regression Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search; (i) DT Model performance on Cross Validation with Grid, Cross Validation without Grid Search, Without Cross validation and Grid Search.

Fig 8. The illustration of proposed clinical application.