Alternatives to Hazard Ratios for Comparing the Efficacy or Safety of Therapies in Noninferiority Studies

Abstract

A noninferiority study is often used to investigate whether a treatment's efficacy or safety profile is acceptable compared with an alternative therapy regarding the time to a clinical event. The empirical quantification of the treatment difference for such a study is routinely based on the hazard ratio (HR) estimate. The HR, which is not a relative risk, may be difficult to interpret clinically, especially when the underlying proportional hazards assumption is violated. The precision of the HR estimate depends primarily on the number of observed events but not directly on exposure times or sample size of the study population. If the event rate is low, the study may require an impractically large number of events to ensure that the prespecified noninferiority criterion for the HR is attainable. This article discusses deficiencies in the current approach for the design and analysis of a noninferiority study. Alternative procedures are provided, which do not depend on any model assumption, to compare 2 treatments. For a noninferiority safety study, the patients' exposure times are more clinically important than the observed number of events. If the patients' exposure times are long enough to evaluate safety reliably, then these alternative procedures can effectively provide clinically interpretable evidence on safety, even with relatively few observed events. These procedures are illustrated with data from 2 studies. One explores the cardiovascular safety of a pain medicine; the second examines the cardiovascular safety of a new treatment for diabetes. These alternative strategies to evaluate safety or efficacy of an intervention lead to more meaningful interpretations of the analysis results than the conventional strategy that uses the HR estimate.

Appendix Figure 1. Graphical presentation of between-group difference metrics

(A) the hazard ratio, (B) the relative time, and (C) various model-free, between-group difference measures for a new treatment (solid curve) and a control (dotted curve); the slopes of the lines in A and B are the hazard ratio and the relative time, respectively, the distance between the two closed circles (horizontal line) in C is the difference for two medians; and the distance between the two closed squares in C (vertical line) is the difference of two event rates at 36-months; the shaded area in C is the difference in the restricted mean survival time up to 48 months

Figure 1

Empirical cumulative incidence curves for patients randomized to celecoxib 400 mg BID (blue, dashed line) and placebo (red solid line) in the celecoxib study (7).

Figure 2

Empirical cumulative incidence curves with reconstructed event time data for the saxagliptin study (9); (A) Saxagliptin arm (solid line) and (B) Placebo arm (dotted line). The shaded area (the area above the cumulative incidence curve) in each panel is the restricted mean survival time up to 900 days

Figure 3

Saxagliptin versus placebo. Point estimate (solid) and 0.95 simultaneous confidence band (dotted) for the difference of the cumulative incidence