A Machine Learning Methodology for Identification and Triage of Heart Failure Exacerbations

Abstract

Inadequate at-home management and self-awareness of heart failure (HF) exacerbations are known to be leading causes of the greater than 1 million estimated HF-related hospitalizations in the USA alone. Most current at-home HF management protocols include paper guidelines or exploratory health applications that lack rigor and validation at the level of the individual patient. We report on a novel triage methodology that uses machine learning predictions for real-time detection and assessment of exacerbations. Medical specialist opinions on statistically and clinically comprehensive, simulated patient cases were used to train and validate prediction algorithms. Model performance was assessed by comparison to physician panel consensus in a representative, out-of-sample validation set of 100 vignettes. Algorithm prediction accuracy and safety indicators surpassed all individual specialists in identifying consensus opinion on existence/severity of exacerbations and appropriate treatment response. The algorithms also scored the highest sensitivity, specificity, and PPV when assessing the need for emergency care. Here we develop a machine-learning approach for providing real-time decision support to adults diagnosed with congestive heart failure. The algorithm achieves higher exacerbation and triage classification performance than any individual physician when compared to physician consensus opinion.

Keywords: Congestive heart failure; Early detection and treatment; Exacerbation, triage; Machine learning; Telehealth monitoring; Triage.

Figure 1

Description of the patient case generation process and splitting into training and validation data

Figure 2

Algorithm training process

Figure 3

Top: Accuracy comparison of the algorithm and the individual physicians at predicting the validation set consensus for triage identification (left). Comparison of the major performance metrics between the algorithm and the average physician (right). The black line represents one standard deviation. Bottom: Comparison of the accuracy (left) and performance metrics (right) of the algorithm against the physicians when member votes are not included in assessing the accuracy. The black line represents one standard deviation

Figure 4

Confusion matrices of the algorithm and the top physician (in terms of accuracy) for a final triage, b exacerbation, and c recommended treatment detection

Figure 5

Distribution of decisions for each physician (left) and averaged physician decision distribution (right), the black line represents 1 standard deviation about the mean. a Triage, b exacerbation, c recommended treatment

Figure 6

Plot of % of triage cases in the validation set that change consensus decision as additional doctors (plus the algorithm) are added to the validation panel. The shaded region around the mean represents one standard deviation. The average change when the panel increases from 8 to 9 members is 5.2%