Precise and accurate power of the rank-sum test for a continuous outcome

Abstract

Accurate power calculations are essential in small studies containing expensive experimental units or high-stakes exposures. Herein, power of the Wilcoxon Mann-Whitney rank-sum test of a continuous outcome is formulated using a Monte Carlo approach and defining [Formula: see text] as a measure of effect size, where [Formula: see text] and [Formula: see text] denote random observations from two distributions hypothesized to be equal under the null. Effect size [Formula: see text] fosters productive communications because researchers understand [Formula: see text] is analogous to a fair coin toss, and [Formula: see text] near 0 or 1 represents a large effect. This approach is feasible even without background data. Simulations were conducted comparing the empirical power approach to existing approaches by Rosner & Glynn, Shieh and colleagues, Noether, and O'Brien-Castelloe. Approximations by Noether and O'Brien-Castelloe are shown to be inaccurate for small sample sizes. The Rosner & Glynn and Shieh, Jan & Randles approaches performed well in many small sample scenarios, though both are restricted to location-shift alternatives and neither approach is theoretically justified for small samples. The empirical method is recommended and available in the R package wmwpow.

Keywords: Mann–Whitney test; Monte Carlo simulation; Wilcoxon rank-sum test; non-parametric; power analysis.

Figure 1:

Power and type I error for the 2-sided Wilcoxon Mann-Whitney rank-sum test, α = 0.05. Sim = empirical simulation. Panel (a) presents power from the empirical Monte Carlo method with a normal distribution and equal standard deviations (k = 1), and panels (b-d) compare power results for a given sample size per group.

Figure 2:

Empirical power for F and G normal with standard deviation ratio k = σ_y/σ_x = 1,2,3,4, α = 0.05.