Best (but oft-forgotten) practices: designing, analyzing, and reporting cluster randomized controlled trials

Abstract

Cluster randomized controlled trials (cRCTs; also known as group randomized trials and community-randomized trials) are multilevel experiments in which units that are randomly assigned to experimental conditions are sets of grouped individuals, whereas outcomes are recorded at the individual level. In human cRCTs, clusters that are randomly assigned are typically families, classrooms, schools, worksites, or counties. With growing interest in community-based, public health, and policy interventions to reduce obesity or improve nutrition, the use of cRCTs has increased. Errors in the design, analysis, and interpretation of cRCTs are unfortunately all too common. This situation seems to stem in part from investigator confusion about how the unit of randomization affects causal inferences and the statistical procedures required for the valid estimation and testing of effects. In this article, we provide a brief introduction and overview of the importance of cRCTs and highlight and explain important considerations for the design, analysis, and reporting of cRCTs by using published examples.

Keywords: community randomized trial; group randomized trial; intraclass correlation coefficient; power analysis; reporting fidelity.

FIGURE 1

Flow diagram to illustrate an example process for a cRCT. Each of the 4 panels from top to bottom highlights different levels of design and analysis considerations. White text boxes indicate design characteristics consistent with cRCTs, whereas black text boxes denote characteristics inconsistent with cRCTs (coupled with the unit being crossed out with an X). A crossed out design does not indicate an invalid study design per se but, rather, indicates a design that is beyond the scope of this article. “A” and “B” in the randomization row represent intervention allocations that would later be assigned to an intervention or control condition. Bubble 1: Non-cRCT design, such as a multisite RCT. Bubble 2: Observation of cluster-level outcomes makes the design no longer a multilevel design. Bubble 3: Analysis of individual-level observations as cluster-level outcomes ignores the multilevel nature of data and requires a different set of considerations for analyzing data than addressed herein. cRCT, cluster randomized controlled trial; RCT, randomized controlled trial.

FIGURE 2

Power curves of an example, 2-armed cRCT as a function of number of clusters and individuals within cluster. Curves show the power to detect a 0.25-unit difference (d) of the outcome with an SD of 1 (i.e., a standardized mean difference of 0.25), ICC of 0.02, and α = 0.05. The calculations assume an equal cluster size (m) and equal allocation (K ÷ 2) between treatments and use a t distribution. Although other values of ICC, d, and SDs may be used, the general shape of the lines will be qualitatively similar under different conditions, with greater K resulting in higher power than with lesser K for the same m. See Appendix A (Additional details: power analysis and sample-size calculation) for sample-size equations. ICC, intraclass correlation.

FIGURE 3

Simulation studies showed that ignoring ICC in analyses can affect the underlying type I error rate. Horizontal dotted lines demark the desired nominal α = 0.05 significance level. A: Grydeland et al. (15) observed an ICC of 0.02 but excluded it from their analyses. They reported a significant difference in the total sample and in a subanalysis of girls only. ICCs ≤0.02 may still affect the type I error. B: Bere et al. (18) did not report the ICC of their clusters; nor did they report an analysis that accounted for clustering. A reasonable range of ICCs from 0.001 to 0.05 were selected, and type I error rates were estimated from 10,000 independent simulated datasets per ICC and illustrated with 95% CIs. See Supplemental Material: Simulation for methods. ICC, intraclass correlation.

FIGURE 4

Ratings of compliance with specific criteria for the CONSORT for cRCTs. For each criterion, articles were assigned “Yes” if it was adequately reported; “Other” if it was partially, incompletely, unclearly, or implicitly reported; or “No” if it was not reported. Only items specific to cRCTs were rated. See Supplemental Material: Compliance for methods. CONSORT, Consolidated Standards of Reporting Trials; cRCT, cluster randomized controlled trial.