Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics

Abstract

Aims: Machine learning (ML) is widely believed to be able to learn complex hidden interactions from the data and has the potential in predicting events such as heart failure (HF) readmission and death. Recent studies have revealed conflicting results likely due to failure to take into account the class imbalance problem commonly seen with medical data. We developed a new ML approach to predict 30 day HF readmission or death and compared the performance of this model with other commonly used prediction models.

Methods and results: We identified all Western Australian patients aged above 65 years admitted for HF between 2003 and 2008 in the linked Hospital Morbidity Data Collection. Taking into consideration the class imbalance problem, we developed a multi-layer perceptron (MLP)-based approach to predict 30 day HF readmission or death and compared the predictive performances using the performance metrics, that is, area under the receiver operating characteristic curve (AUC), area under the precision-recall curve (AUPRC), sensitivity and specificity with other ML and regression models. Out of the 10 757 patients with HF, 23.6% were readmitted or died within 30 days of hospital discharge. We observed an AUC of 0.55, 0.53, 0.58, and 0.54 while an AUPRC of 0.39, 0.38, 0.46, and 0.38 for weighted random forest, weighted decision trees, logistic regression, and weighted support vector machines models, respectively. The MLP-based approach produced the highest AUC (0.62) and AUPRC (0.46) with 48% sensitivity and 70% specificity.

Conclusions: We show that for the medical data with class imbalance, the proposed MLP-based approach is superior to other ML and regression techniques for the prediction of 30 day HF readmission or death.

Keywords: Heart failure; Machine learning; Prediction; Readmission.

Figure 1

Cohort identification flow chart. The final cohort contains 10 757 HF patients admitted to WA hospitals in 2003–2008. HF, Heart failure; WA, Western Australia.

Figure 2

Our three‐layer multi‐layer perceptron network with one hidden layer ‘h’ containing 50 hidden nodes. ‘W’ represents the node weights, ‘b’ denotes the node biases, ‘x’ is the input feature vector, and ‘y’ represents the binary output.