Comparisons against baseline within randomised groups are often used and can be highly misleading

Abstract

Background: In randomised trials, rather than comparing randomised groups directly some researchers carry out a significance test comparing a baseline with a final measurement separately in each group.

Methods: We give several examples where this has been done. We use simulation to demonstrate that the procedure is invalid and also show this algebraically.

Results: This approach is biased and invalid, producing conclusions which are, potentially, highly misleading. The actual alpha level of this procedure can be as high as 0.50 for two groups and 0.75 for three.

Conclusions: Randomised groups should be compared directly by two-sample methods and separate tests against baseline are highly misleading.

Figure 1

Actual Type I error (alpha) for separate tests against baseline for two randomised groups against the power of the individual test of change from baseline.

Figure 2

Actual Type I error (alpha) for separate tests against baseline for three randomised groups against the power of the individual test of change from baseline.