Simple, defensible sample sizes based on cost efficiency

Abstract

The conventional approach of choosing sample size to provide 80% or greater power ignores the cost implications of different sample size choices. Costs, however, are often impossible for investigators and funders to ignore in actual practice. Here, we propose and justify a new approach for choosing sample size based on cost efficiency, the ratio of a study's projected scientific and/or practical value to its total cost. By showing that a study's projected value exhibits diminishing marginal returns as a function of increasing sample size for a wide variety of definitions of study value, we are able to develop two simple choices that can be defended as more cost efficient than any larger sample size. The first is to choose the sample size that minimizes the average cost per subject. The second is to choose sample size to minimize total cost divided by the square root of sample size. This latter method is theoretically more justifiable for innovative studies, but also performs reasonably well and has some justification in other cases. For example, if projected study value is assumed to be proportional to power at a specific alternative and total cost is a linear function of sample size, then this approach is guaranteed either to produce more than 90% power or to be more cost efficient than any sample size that does. These methods are easy to implement, based on reliable inputs, and well justified, so they should be regarded as acceptable alternatives to current conventional approaches.

Figure 1

Shapes of the relation between projected value and sample size for 10 measures of study value and situations. For visual clarity and because only the shapes are of interest, the vertical scale varies for different curves. Shown are curves for value proportional to: a) gain in Shannon information with n₀=100, where n₀ is the sample size equivalent of the prior information; b) reciprocal of confidence interval width; c) reduction in Bayesian credible interval width when n₀=100; d) reduction in squared error versus using prior mean when n₀=100; e) power for a standardized effect size of 0.2; f) additional cures from a Bayesian clinical trial with prior means (SDs) for cure rates of 0.4 (0.05) versus 0.4 (0.1); g) gain in Shannon information with n₀=2; h) reduction in squared error versus using a single observation; i) reduction in squared error versus using prior mean when n₀=2; j) reduction in Bayesian credible interval width when n₀=2.