Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials

Abstract

The intraclass correlation coefficient (ICC) is a fundamental parameter of interest in cluster randomized trials as it can greatly affect statistical power. We compare common methods of estimating the ICC in cluster randomized trials with binary outcomes, with a specific focus on their application to community-based cancer prevention trials with primary outcome of self-reported cancer screening. Using three real data sets from cancer screening intervention trials with different numbers and types of clusters and cluster sizes, we obtained point estimates and 95% confidence intervals for the ICC using five methods: the analysis of variance estimator, the Fleiss-Cuzick estimator, the Pearson estimator, an estimator based on generalized estimating equations and an estimator from a random intercept logistic regression model. We compared estimates of the ICC for the overall sample and by study condition. Our results show that ICC estimates from different methods can be quite different, although confidence intervals generally overlap. The ICC varied substantially by study condition in two studies, suggesting that the common practice of assuming a common ICC across all clusters in the trial is questionable. A simulation study confirmed pitfalls of erroneously assuming a common ICC. Investigators should consider using sample size and analysis methods that allow the ICC to vary by study condition.

Figure 1

Distribution of cluster-level proportions in the three data sets

Figure 2

Results of Simulation Study 2 to compare point estimates of overall ICC from five ICC estimation methods (ano, ANOVA; fc, Fleiss-Cusick; pe, Pearson; gee, generalized estimating equations; re, random intercept logistic regression) when data arise from a two-arm trial with same π but different ρ in each arm using 2000 simulated data sets for each scenario

Figure 3

Results of Simulation Study 2 to compare point estimates of overall ICC from five ICC estimation methods (ano, ANOVA; fc, Fleiss-Cusick; pe, Pearson; gee, generalized estimating equations; re, random intercept logistic regression) when data arise from a two-arm trial with same ρ but different π in each arm using 2000 simulated data sets for each scenario

Figure 4

Results of Simulation Study 2 to compare point estimates of overall ICC from five ICC estimation methods (ano, ANOVA; fc, Fleiss-Cusick; pe, Pearson; gee, generalized estimating equations; re, random intercept logistic regression) when data arise from a two-arm trial with different ρ and different π in each arm using 2000 simulated data sets for each scenario