Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials

Abstract

Objective: To determine the statistical power to detect potentially clinically significant differences in serious adverse events between drug therapies reported in a sample of randomized controlled trials (RCTs).

Study design and setting: Systematic review of RCTs with positive efficacy endpoint and at least a twofold difference in the proportion of patients with serious adverse events between treatment groups from six major journals. The power of each study to detect statistically significant differences in serious adverse events was calculated.

Results: Of the six included trials, all performed statistical analysis on adverse events without disclosure of the statistical power for detecting the reported differences between groups. The power of each study to detect the reported differences in adverse events was calculated and yielded values ranging from 0.07 to 0.37 among trials with non-statistically significant differences.

Conclusion: Statistical testing for differences in the proportion of patients experiencing an adverse event is common in RCTs; non-statistically significant differences are associated with low statistical power. A high probability of type II error may lead to erroneous clinical inference resulting in harm. The statistical power for nonsignificant tests should be considered in the interpretation of results.