Considerable variation in NNT - a study based on Monte Carlo simulations

Abstract

Objective: The aim of this analysis was to explore the variation in measures of effect, such as the number-needed-to-treat (NNT) and the relative risk (RR).

Study design and setting: We performed Monte Carlo simulations of therapies using binominal distributions based on different true absolute risk reductions (ARR), number of patients (n), and the baseline risk of adverse events (p(0)) as parameters and presented results in histograms with NNT and RR. We also estimated the probability of observing no or a negative treatment effect, given that the true effect is positive.

Results: When RR is used to express treatment effectiveness, it has a regular distribution around the expected value for various values of true ARR, n, and p(0). The equivalent distribution of NNT is by definition nonconnected at zero and is also irregular. The probability that the observed treatment effectiveness is zero or negative when the true value is positive depends on n, p(0), and the true ARR. In some cases, this probability is even higher than 50%.

Conclusion: For realistic values of true ARR, n, and p(0), the observed NNT varies much more than the observed ARR and RR. Clinicians should use NNT cautiously when expressing treatment benefits.