Intraclass correlations from a community-based alcohol prevention study: the effect of repeat observations on the same communities

Abstract

Objective: School- and community-based alcohol prevention programs are often evaluated using a group-randomized trial (GRT) design with a single pretest and a single posttest survey. To size such studies properly, investigators need accurate estimates of the variance and intraclass correlation that will be operative in their analyses. Until recently, the only available estimates were based on cross-sectional analyses. A recent report suggests that values from cross-sectional analyses may overestimate the intraclass correlation operative in pretest-posttest analyses. The purpose of this article is to review these issues, present estimates of intraclass correlation for a variety of alcohol-related endpoints based on cross-sectional analyses and to compare those estimates to estimates based on pretest-posttest analyses. We will also show how these estimates can be used to establish optimal sample sizes for GRTs to evaluate school- and community-based alcohol prevention programs.

Method: Data were collected from 18 to 20 year olds and high-school seniors as part of an alcohol prevention effort employing a group-randomized trial design with a single pretest and a single posttest survey. Data were analyzed via mixed-model regression methods to estimate components of variance. Those components were then used to compute the intraclass correlations operative in both cross-sectional analyses and in pretest-posttest analyses.

Results: Results indicate that intraclass correlations operative in pretest-posttest analyses are much smaller than are those operative in cross-sectional analyses.

Conclusions: Our findings indicate that future alcohol-prevention studies employing a group-randomized trial design with a single pretest and single posttest survey may not need to be as large as previously suggested by intraclass correlation estimates based on cross-sectional data. This holds true even if they are analyzed to reflect the extra variation typical of group-randomized trials.