How large are the consequences of covariate imbalance in cluster randomized trials: a simulation study with a continuous outcome and a binary covariate at the cluster level

Abstract

Background: The number of clusters in a cluster randomized trial is often low. It is therefore likely random assignment of clusters to treatment conditions results in covariate imbalance. There are no studies that quantify the consequences of covariate imbalance in cluster randomized trials on parameter and standard error bias and on power to detect treatment effects.

Methods: The consequences of covariance imbalance in unadjusted and adjusted linear mixed models are investigated by means of a simulation study. The factors in this study are the degree of imbalance, the covariate effect size, the cluster size and the intraclass correlation coefficient. The covariate is binary and measured at the cluster level; the outcome is continuous and measured at the individual level.

Results: The results show covariate imbalance results in negligible parameter bias and small standard error bias in adjusted linear mixed models. Ignoring the possibility of covariate imbalance while calculating the sample size at the cluster level may result in a loss in power of at most 25 % in the adjusted linear mixed model. The results are more severe for the unadjusted linear mixed model: parameter biases up to 100 % and standard error biases up to 200 % may be observed. Power levels based on the unadjusted linear mixed model are often too low. The consequences are most severe for large clusters and/or small intraclass correlation coefficients since then the required number of clusters to achieve a desired power level is smallest.

Conclusions: The possibility of covariate imbalance should be taken into account while calculating the sample size of a cluster randomized trial. Otherwise more sophisticated methods to randomize clusters to treatments should be used, such as stratification or balance algorithms. All relevant covariates should be carefully identified, be actually measured and included in the statistical model to avoid severe levels of parameter and standard error bias and insufficient power levels.

Keywords: Adjusted linear mixed model; Cluster randomization; Covariate imbalance; Simulation study; Unadjusted linear mixed model.

Fig. 1

Parameter bias for the linear mixed model. Legend: The degree of covariate imbalance is given on the horizontal axes and expressed as quantiles of the hypergeometric distribution. Values below 0.5 imply negative covariate imbalance, values above 0.5 imply positive covariate imbalance and the value 0.5 implies covariate balance

Fig. 2

Standard error bias for the linear mixed model. Legend: The degree of covariate imbalance is given on the horizontal axes and expressed as quantiles of the hypergeometric distribution. Values below 0.5 imply negative covariate imbalance, values above 0.5 imply positive covariate imbalance and the value 0.5 implies covariate balance

Fig. 3

Empirical power levels for the adjusted linear mixed model. Legend: The degree of covariate imbalance is given on the horizontal axes and expressed as quantiles of the hypergeometric distribution. Values below 0.5 imply negative covariate imbalance, values above 0.5 imply positive covariate imbalance and the value 0.5 implies covariate balance. The horizontal solid line represents the nominal power level

Fig. 4

Empirical power levels for the linear mixed model. Legend: The degree of covariate imbalance is given on the horizontal axes and expressed as quantiles of the hypergeometric distribution. Values below 0.5 imply negative covariate imbalance, values above 0.5 imply positive covariate imbalance and the value 0.5 implies covariate balance