Analysis of data from group-randomized trials with repeat observations on the same groups

Abstract

This study used Monte Carlo simulations to evaluate the performance of alternative models for the analysis of group-randomized trials having more than two time intervals for data collection. The major distinction among the models tested was the sampling variance of the intervention effect. In the mixed-model ANOVA, the sampling variance of the intervention effect is based on the variance among group x time-interval means. In the random coefficients model, the sampling variance of the intervention effect is based on the variance among the group-specific slopes. These models are equivalent when the design includes only two time intervals, but not when there are more than two time intervals. The results indicate that the mixed-model ANOVA yields unbiased estimates of sampling variation and nominal type I error rates when the group-specific time trends are homogenous. However, when the group-specific time trends are heterogeneous, the mixed-model ANOVA yields downwardly biased estimates of sampling variance and inflated type I error rates. In contrast, the random coefficients model yields unbiased estimates of sampling variance and the nominal type I error rate regardless of the pattern among the groups. We discuss implications for the analysis of group-randomized trials with more than two time intervals.