Assessment of Noninferiority Margins in Cardiovascular Medicine Trials

Abstract

Background: Noninferiority trials are increasingly common in cardiovascular medicine, but their reporting and interpretation are challenging, particularly when an absolute risk difference is used as noninferiority margin.

Objectives: This study aimed to investigate the effect of using absolute rather than relative noninferiority margins in cardiovascular trials.

Methods: We reviewed noninferiority trials presented at major cardiovascular conferences from 2015 to 2022 and published within the same period. Based on the actual versus anticipated event rates in the control group, we recalculated the absolute noninferiority margin and re-assessed the trial results. The primary outcome of interest was the proportion of trials with a different interpretation after recalculation. Additionally, we analyzed the conclusion statements of these trials to determine if cautionary notes for the interpretation of study results were included.

Results: We analyzed a total of 768 trials, of which 88 had a noninferiority design and 66 used an absolute noninferiority margin. Of 48 comparisons from 45 trials qualifying for the analysis, 11 (22.9%) had divergent results after recalculation of the absolute noninferiority margin based on the observed rather than anticipated event rate. Ten trials originally claiming noninferiority, did not meet it after the margin recalculation. All of them did not include statements suggesting cautionary interpretation of the study results in the conclusion section. Compared with the other trials, these displayed a larger median difference between anticipated and recalculated noninferiority margins (44.7% [IQR: 38.6%-56.7%] vs 15.3% [IQR: -1.5% to 28.9%]; P < 0.001).

Conclusions: Recalculating noninferiority margins based on actual event rates, rather than anticipated ones, led to different outcomes in approximately 1 out of 4 cardiovascular trials, with most divergent trials lacking cautionary interpretation. These findings emphasize the importance of using or supplementing the relative noninferiority margin, particularly in studies with significant deviations between observed and expected event rates. This underscores the critical need for enhanced methodological and reporting standards in noninferiority trials, especially those employing absolute margins.

Keywords: cardiovascular medicine; methodology; noninferiority; noninferiority margin; randomized trials; trial interpretation.

Graphical abstract

Figure 1

Study Flow Chart ACC = American College of Cardiology; AHA = American Heart Association; ESC = European Society of Cardiology; LBS = Late Breaking Science; NI = noninferiority; RCT = randomized controlled trial; TCT = Transcatheter Cardiovascular Therapeutics.

Central Illustration

Assessment of Noninferiority Margins in Cardiovascular Medicine Trials Distribution of noninferiority randomized controlled trials with relative or absolute margin over the eligible population of studies. Number of included analyses who have met noninferiority before (eg, according to authors) and after the recalculation of noninferiority margin; Number prevalence of analyses not including cautionary notes of interpretation among studies with divergent results after the noninferiority margin recalculation. ∗Indicates 1 single trial that did not claim noninferiority in the original analysis, but met noninferiority criteria after recalculation, therefore being excluded from the assessment of study conclusions. ARD = absolute risk difference; NI = noninferiority; RRR = relative risk ratio.

Figure 2

Trials Showing Divergent Results After the Recalculation of Noninferiority Margin Dashed orange lines refer to noninferiority margins based on the anticipated event rate in the control group; solid orange lines refer to recalculated noninferiority margins based on the observed event rate in the control group. Blue circles and lines represent absolute risk difference and CIs; light yellow and light blue stands for treatment better and control better areas, respectively. Circ Int = Circulation: Cardiovascular Interventions; EHJ = European Heart Journal; EIJ = EuroIntervention Journal; JACC Intv = JACC: Cardiovascular Interventions; JAMA = Journal of the American Medical Association; NEJM = New England Journal of Medicine; NI = noninferiority.