Prediction of mortality in heart failure by machine learning. Comparison with statistical modeling

Abstract

Background: Assessing the relative performance of machine learning (ML) methods and conventional statistical methods in predicting prognosis in heart failure (HF) still remains a challenging research field.

Methods: The primary outcome was three-year mortality. The following 5 machine learning approaches were used for modeling: Random Forest (RF), Gradient Boosting, Extreme Gradient Boosting (XGBoost), Support Vector Machine, and Multilayer perceptron. We compared the performance of the best performing ML models to the MAGGIC (Meta-analysis Global Group in Chronic Heart Failure) score and a novel logistic regression model (LRM) developed using the same set of variables used to develop the machine learning models. The performance was determined based on discrimination, calibration, and net benefit.

Results: The XGBoost and the RF were the best performing ML models. The XGBoost provided the highest discrimination (C-statistic: 0.793) and the lowest Brier score (0.178); the RF model had a C-statistic of 0.779 and provided the highest area under the precision-recall curve (0.636). Both models were well calibrated. Both the XGboost and RF models outperformed MAGGIC score. The LRM had a C-statistic of 0.811 and a Brier score of 0.160 and was well calibrated. The XGBoost, RF, and LRM gave a higher net benefit than MAGGIC score; the XGBoost, RF, and logistic regression model gave similar net benefit.

Conclusions: RF and XGBoost models outperformed MAGGIC in predicting mortality. However, they did not offer any improvement over a logistic regression model built using the same set of covariates considered in the ML modeling.

Keywords: Heart failure; Machine learning modeling; Mortality; Risk prediction; Statistical modeling.