Augmented Intelligence to Identify Patients With Advanced Heart Failure in an Integrated Health System

Abstract

Background: Timely referral for specialist evaluation in patients with advanced heart failure (HF) is a Class 1 recommendation. However, the transition from stage C HF to advanced or stage D HF often goes undetected in routine care, resulting in delayed referral and higher mortality rates.

Objectives: The authors sought to develop an augmented intelligence-enabled workflow using machine learning to identify patients with stage D HF and streamline referral.

Methods: We extracted data on HF patients with encounters from January 1, 2007, to November 30, 2020, from a HF registry within a regional, integrated health system. We created an ensemble machine learning model to predict stage C or stage D HF and integrated the results within the electronic health record.

Results: In a retrospective data set of 14,846 patients, the model had a good positive predictive value (60%) and low sensitivity (25%) for identifying stage D HF in a 100-person, physician-reviewed, holdout test set. During prospective implementation of the workflow from April 1, 2021, to February 15, 2022, 416 patients were reviewed by a clinical coordinator, with agreement between the model and the coordinator in 50.3% of stage D predictions. Twenty-four patients have been scheduled for evaluation in a HF clinic, 4 patients started an evaluation for advanced therapies, and 1 patient received a left ventricular assist device.

Conclusions: An augmented intelligence-enabled workflow was integrated into clinical operations to identify patients with advanced HF. Endeavors such as this require a multidisciplinary team with experience in design thinking, informatics, quality improvement, operations, and health information technology, as well as dedicated resources to monitor and improve performance over time.

Keywords: advanced heart failure; artificial intelligence; augmented intelligence; electronic health record; integrated healthcare system; machine learning.

Graphical abstract

Figure 1

Cohort Generation The schematic is shown for the generation of the cohort used to train the machine learning algorithm. HF = heart failure.

Figure 2

Model Development and Implementation A description of the model development and implementation using cloud computing via Microsoft Azure. EHR = electronic health record; HF = heart failure.

Figure 3

Augmented Intelligence-Enabled Clinical Workflow A description of the augmented intelligence-enabled clinical workflow within the electronic health record. HF = heart failure.

Central Illustration

The Use of Augmented Intelligence to Identify Patients With Stage D Heart Failure A depiction of the steps involved in data acquisition, creation of the machine learning model, and the augmented intelligence-based clinical workflow utilizing the model is shown. Patients for training set generation were identified within the Healthy Planet Heart Failure Registry within epic. Data were collected from an enterprise data warehouse associated with a large, integrated health system, as well as from the electronic health record itself. An ensemble machine learning model was created, consisting of 2 gradient boosting trees as well as a feedforward neural network, with the final classification made by vote score. At the time of inference, the model predicts whether a patient has stage D HF and that prediction is embedded within a workflow in the electronic health record. This is subsequently reviewed by a nurse coordinator, and clinician outreach for appropriate patients is performed to streamline access to heart failure clinic for further evaluation. The labels generated by the nurse coordinator are saved for future iterations of model training. EHR = electronic health record; HF = heart failure; ML = machine learning.