Benchmarking Observational Analyses Before Using Them to Address Questions Trials Do Not Answer: An Application to Coronary Thrombus Aspiration

Abstract

To increase confidence in the use of observational analyses when addressing effectiveness questions beyond those addressed by randomized trials, one can first benchmark the observational analyses against existing trial results. We used Swedish registry data to emulate a target trial similar to the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) randomized trial, which found no difference in the risk of death or myocardial infarction by 1 year with or without thrombus aspiration among individuals with ST-elevation myocardial infarction. We benchmarked the emulation against the trial at 1 year and then extended the emulation's follow-up to 3 years and estimated effects in subpopulations underrepresented in the trial. As in the TASTE trial, the observational analysis found no differences in risk of outcomes by 1 year between groups (risk difference = 0.7 (confidence interval, -0.7, 2.0) and -0.2 (confidence interval, -1.3, 1.0) for death and myocardial infarction, respectively), so benchmarking was considered successful. We additionally showed no difference in risk of death or myocardial infarction by 3 years, or within subpopulations by 1 year. Benchmarking against an index trial before using observational analyses to answer questions beyond those the trial could address allowed us to explore whether the observational data can be trusted to deliver valid estimates of treatment effects.

Keywords: benchmarking; causal inference; observational analyses; randomized trial; target trial emulation.

Figure 1

Flowchart of individuals eligible for an observational emulation of a target trial of thrombus aspiration versus no thrombus aspiration, Swedish Web-Based System for Enhancement and Development of Evidence-Based Care in Heart Disease Evaluated According to Recommended Therapies (SWEDEHEART) registry, Sweden, 2007–2016. There were 18,222 eligible individuals, of whom 3,462 were given thrombus aspiration and 14,760 were not given thrombus aspiration. CABG, coronary artery bypass graft; PCI, percutaneous coronary intervention; STEMI, ST-elevation myocardial infarction; TASTE, Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia.

Figure 2

Standardized survival curves for outcomes of death (A) and myocardial infarction (B) from an observational emulation of a target trial of thrombus aspiration versus no thrombus aspiration, Swedish Web-Based System for Enhancement and Development of Evidence-Based Care in Heart Disease Evaluated According to Recommended Therapies (SWEDEHEART) registry, Sweden, 2007–2016. The 3-year risk difference for death was 0.9% (95% confidence interval: −0.7, 2.4), and the 3-year risk difference of myocardial infraction was −0.2% (95% confidence interval: −1.5, 1.1).