Prospective Validation and Comparative Analysis of Coronary Risk Stratification Strategies Among Emergency Department Patients With Chest Pain

Abstract

Background Coronary risk stratification is recommended for emergency department patients with chest pain. Many protocols are designed as "rule-out" binary classification strategies, while others use graded-risk stratification. The comparative performance of competing approaches at varying levels of risk tolerance has not been widely reported. Methods and Results This is a prospective cohort study of adult patients with chest pain presenting between January 2018 and December 2019 to 13 medical center emergency departments within an integrated healthcare delivery system. Using an electronic clinical decision support interface, we externally validated and assessed the net benefit (at varying risk thresholds) of several coronary risk scores (History, ECG, Age, Risk Factors, and Troponin [HEART] score, HEART pathway, Emergency Department Assessment of Chest Pain Score Accelerated Diagnostic Protocol), troponin-only strategies (fourth-generation assay), unstructured physician gestalt, and a novel risk algorithm (RISTRA-ACS). The primary outcome was 60-day major adverse cardiac event defined as myocardial infarction, cardiac arrest, cardiogenic shock, coronary revascularization, or all-cause mortality. There were 13 192 patient encounters included with a 60-day major adverse cardiac event incidence of 3.7%. RISTRA-ACS and HEART pathway had the lowest negative likelihood ratios (0.06, 95% CI, 0.03-0.10 and 0.07, 95% CI, 0.04-0.11, respectively) and the greatest net benefit across a range of low-risk thresholds. RISTRA-ACS demonstrated the highest discrimination for 60-day major adverse cardiac event (area under the receiver operating characteristic curve 0.92, 95% CI, 0.91-0.94, P<0.0001). Conclusions RISTRA-ACS and HEART pathway were the optimal rule-out approaches, while RISTRA-ACS was the best-performing graded-risk approach. RISTRA-ACS offers promise as a versatile single approach to emergency department coronary risk stratification. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT03286179.

Keywords: acute coronary syndrome; emergency department; risk score.

Figure 1. RISTRA‐ACS algorithm.

Upon data entry into the electronic clinical decision support interface (RISTRA), risk estimation begins with calculation of both the HEART score and the EDACS (green boxes). Subsequent risk adjustment is based on the peak troponin value (purple boxes) obtained at least 3 hours from symptom onset (if both HEART and EDACS scores indicate low risk) or at least 4.5 hours from pain onset (if either HEART or EDACS scores indicate non‐low risk). A minimum risk override was implemented in the presence of crescendo angina (yellow box, minimal risk 3%) or new ischemic ECG changes (>7% risk), with higher risk assignment allowed if criteria were present. Risk estimates were presented to the clinician following completion of data entry and troponin testing completion, indicated by either a stable or decreasing troponin value on repeated measures at least 2 hours apart or a single measure below the LOQ at or beyond the above specified time from pain onset. EDACS indicates Emergency Department Assessment of Chest Pain Score; HEART, History, Electrocardiogram, Age, Risk Factors, Troponin; LOQ, limit of quantitation; MACE, major adverse cardiac event at 60 days; RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome; and TnI, troponin I.

Figure 2. Study cohort flow chart.

eCDS indicates electronic clinical decision support; ED, emergency department; KPNC, Kaiser Permanente Northern California; MACE, major adverse cardiac event; RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome; and STEMI, ST‐segment–elevation myocardial infarction.

Figure 3. Smoothed calibration plot comparing graded‐risk approaches for the primary outcome of 60‐day MACE.

Approaches are: (1) physician gestalt (Gestalt), (2) HEART score (HEART), (3) troponin strata (Troponin), and (4) RISTRA‐ACS (RISTRA). Calibration was determined using a random split of the study cohort into equal portions to generate sample 1 (testing data set) and sample 2 (validation data set). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE, major adverse cardiac events; and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.

Figure 4. Smoothed calibration plot comparing graded‐risk approaches for the secondary outcome of 60‐day MACE‐CR.

Approaches are: (1) physician gestalt for acute coronary syndrome (Gestalt), (2) HEART score (HEART), (3) troponin strata (Troponin), and (4) RISTRA‐ACS (RISTRA). Calibration was determined using a random split of the study cohort into equal portions to generate sample 1 (testing data set) and sample 2 (validation data set). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE‐CR, major adverse cardiac events minus coronary revascularization; and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.

Figure 5. Decision curves for rule‐out approaches and 60‐day MACE at (A) the observed MACE prevalence of 3.7% and (B) a theoretical MACE prevalence of 10%.

The horizontal axis is restricted to risk thresholds between 0% and 2% to represent a range of low‐risk definitions. Approaches are: (1) troponin below the level of quantitation (Troponin), (2) Emergency Department Assessment of Chest Pain Score Accelerated Diagnostic Protocol (EDACS‐ADP low risk), (3) HEART score <4 (HEART score), (4) HEART Pathway low risk (HEART Pathway), (5) physician gestalt for acute coronary syndrome ≤2% (Gestalt), and (6) very‐low‐risk categorization by RISTRA‐ACS (RISTRA‐ACS). The “test all” line represents an approach in which all patients undergo further testing (ie, a strategy with 100% sensitivity and 0% specificity). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE, major adverse cardiac events and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.

Figure 6. Decision curves for graded‐risk approaches and the primary outcome of 60‐day MACE.

The horizontal axis is restricted to risk thresholds between 0% and 10% to represent a range of low‐to‐moderate‐risk definitions. Approaches are: (1) physician gestalt for acute coronary syndrome (Gestalt), (2) HEART score (HEART), (3) troponin strata (Troponin), and (4) RISTRA‐ACS (RISTRA). The “test all” line represents an approach in which all patients undergo further testing (ie, a strategy with 100% sensitivity and 0% specificity). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE, major adverse cardiac events; and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.

Figure 7. Decision curves for rule‐out approaches and 60‐day MACE‐CR at (A) the observed MACE‐CR prevalence of 3.3% and (B) a theoretical MACE‐CR prevalence of 10%.

The horizontal axis is restricted to risk thresholds between 0% and 2% to represent a range of low‐risk definitions. Approaches are: (1) troponin below the level of quantitation (Troponin), (2) Emergency Department Assessment of Chest Pain Score Accelerated Diagnostic Protocol (EDACS‐ADP low risk), (3) HEART score <4 (HEART score), (4) HEART Pathway low risk (HEART Pathway), (5) physician gestalt for acute coronary syndrome ≤2% (Gestalt) and (6) very‐low‐risk categorization by RISTRA‐ACS (RISTRA‐ACS). The “test all” line represents an approach in which all patients undergo further testing (ie, a strategy with 100% sensitivity and 0% specificity). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE‐CR, major adverse cardiac events without revascularization; and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.

Figure 8. Decision curves for graded‐risk approaches and the secondary outcome of 60‐day MACE‐CR.

The horizontal axis is restricted to risk thresholds between 0% and 10% to represent a range of low‐to‐moderate‐risk definitions. Approaches are: (1) physician gestalt for acute coronary syndrome (Gestalt), (2) HEART score (HEART), (3) troponin strata (Troponin), and (4) RISTRA‐ACS (RISTRA). The “test all” line represents an approach in which all patients undergo further testing (ie, a strategy with 100% sensitivity and 0% specificity). HEART indicates History, Electrocardiogram, Age, Risk Factors, Troponin; MACE‐CR, major adverse cardiac events without revascularization; and RISTRA‐ACS, Risk Stratification–Acute Coronary Syndrome.