Development and validation of a machine learned algorithm to IDENTIFY functionally significant coronary artery disease

Abstract

Introduction: Multiple trials have demonstrated broad performance ranges for tests attempting to detect coronary artery disease. The most common test, SPECT, requires capital-intensive equipment, the use of radionuclides, induction of stress, and time off work and/or travel. Presented here are the development and clinical validation of an office-based machine learned algorithm to identify functionally significant coronary artery disease without radiation, expensive equipment or induced patient stress.

Materials and methods: The IDENTIFY trial (NCT03864081) is a prospective, multicenter, non-randomized, selectively blinded, repository study to collect acquired signals paired with subject meta-data, including outcomes, from subjects with symptoms of coronary artery disease. Time synchronized orthogonal voltage gradient and photoplethysmographic signals were collected for 230 seconds from recumbent subjects at rest within seven days of either left heart catheterization or coronary computed tomography angiography. Following machine learning on a proportion of these data (N = 2,522), a final algorithm was selected, along with a pre-specified cut point on the receiver operating characteristic curve for clinical validation. An unseen set of subject signals (N = 965) was used to validate the algorithm.

Results: At the pre-specified cut point, the sensitivity for detecting functionally significant coronary artery disease was 0.73 (95% CI: 0.68-0.78), and the specificity was 0.68 (0.62-0.74). There exists a point on the receiver operating characteristic curve at which the negative predictive value is the same as coronary computed tomographic angiography, 0.99, assuming a disease incidence of 0.04, yielding sensitivity of 0.89 and specificity of 0.42. Selecting a point at which the positive predictive value is maximized, 0.12, yields sensitivity of 0.39 and specificity of 0.88.

Conclusion: The performance of the machine learned algorithm presented here is comparable to common tertiary center testing for coronary artery disease. Employing multiple cut points on the receiver operating characteristic curve can yield the negative predictive value of coronary computed tomographic angiography and a positive predictive value approaching that of myocardial perfusion imaging. As such, a system employing this algorithm may address the need for a non-invasive, no radiation, no stress, front line test, and hence offer significant advantages to the patient, their physician, and healthcare system.

Keywords: artificial intelligence; coronary artery disease; digital health; front line testing; machine learning (ML).

FIGURE 1

Diagram showing the composition of population A: Sensitivity test group and population B: Specificity test group derived from the IDENTIFY clinical study.

FIGURE 2

(Left) feature coefficients normalized by feature averages and its cumulative sum, and (right) the normalized feature coefficients for the top 10 features. Visual-PPG — features derived from analyzing the PPG and its first and second derivatives in phase space. Wavelet-PPG — features derived from wavelet analysis of PPG signal. PPG-PSD — features deviates from power spectral density analysis of the PPG signal. RCA, repolarization conduction abnormality; analysis of the ventricular repolarization waveform in band-pass limited frequency ranges. DCA, depolarization conduction abnormality; analysis of the ventricular depolarization waveform in band-pass limited frequency ranges. Wavelet-OVG — features derived from wavelet analysis of OVG signal.

FIGURE 3

Coronary artery disease (CAD) model performance on the training dataset by gender.

FIGURE 4

Coronary artery disease (CAD) model performance on the training dataset by gender when adding the influence of age.

FIGURE 5

Consort diagram of Group 2 (Population A and Population B) validation subjects.

FIGURE 6

Consort Diagram of Group 4 validation subjects for specificity. *n = 3 with unknown treatment referral.

FIGURE 7

ROC curve for the model against the validation population.

FIGURE 8

Flow of subjects in a hypothetical population of 10,000 individuals with new onset symptoms of CAD, assuming a pre-test prevalence of 0.04. In the first pass, the machine learned algorithm presented here is used to call individuals as negative for significant CAD if their score is lower than –0.07, and likely positive for functionally significant CAD if their score is greater than 0.1. The group in the middle are secondarily assessed using coronary CTA and SPECT to determine additional subjects that are unlikely to have significant CAD (coronary CTA), or likely to have significant CAD (SPECT). TN, true negative; FN, false negative; TP, true positive; FP, false positive; NPV, negative predictive value; PPV, positive predictive value.