A sample size planning approach that considers both statistical significance and clinical significance

Abstract

Background: The CONSORT statement requires clinical trials to report confidence intervals, which help to assess the precision and clinical importance of the treatment effect. Conventional sample size calculations for clinical trials, however, only consider issues of statistical significance (that is, significance level and power).

Method: A more consistent approach is proposed whereby sample size planning also incorporates information on clinical significance as indicated by the boundaries of the confidence limits of the treatment effect.

Results: The probabilities of declaring a "definitive-positive" or "definitive-negative" result (as defined by Guyatt et al., CMAJ 152(2):169-173, 1995) are controlled by calculating the sample size such that the lower confidence limit under H 1 and the upper confidence limit under H 0 are bounded by relevant cut-offs. Adjustments to the traditional sample size can be directly derived for the comparison of two normally distributed means in a test of nonequality, while simulations are used to estimate the sample size for evaluating the hazards ratio in a proportional-hazards model.

Conclusions: This sample size planning approach allows for an assessment of the potential clinical importance and precision of the treatment effect in a clinical trial in addition to considerations of statistical power and type I error.

Figure 1

Plot of Pr(LCL > kδ | H ₁) (red curve) and Pr(UCL < kδ | H ₀) (blue curve) for k ∈ [0,1], α = 0.05, β = 0.80 in a comparison of normally distributed mean responses with known σ between treatment and control groups for a 1:1 randomized clinical trial.