Evaluating tests for cluster-randomized trials with few clusters under generalized linear mixed models with covariate adjustment: A simulation study

Abstract

Generalized linear mixed models (GLMM) are commonly used to analyze clustered data, but when the number of clusters is small to moderate, standard statistical tests may produce elevated type I error rates. Small-sample corrections have been proposed for continuous or binary outcomes without covariate adjustment. However, appropriate tests to use for count outcomes or under covariate-adjusted models remains unknown. An important setting in which this issue arises is in cluster-randomized trials (CRTs). Because many CRTs have just a few clusters (eg, clinics or health systems), covariate adjustment is particularly critical to address potential chance imbalance and/or low power (eg, adjustment following stratified randomization or for the baseline value of the outcome). We conducted simulations to evaluate GLMM-based tests of the treatment effect that account for the small (10) or moderate (20) number of clusters under a parallel-group CRT setting across scenarios of covariate adjustment (including adjustment for one or more person-level or cluster-level covariates) for both binary and count outcomes. We find that when the intraclass correlation is non-negligible ( $\ge$ 0.01) and the number of covariates is small ( $\le$ 2), likelihood ratio tests with a between-within denominator degree of freedom have type I error rates close to the nominal level. When the number of covariates is moderate ( $\ge$ 5), across our simulation scenarios, the relative performance of the tests varied considerably and no method performed uniformly well. Therefore, we recommend adjusting for no more than a few covariates and using likelihood ratio tests with a between-within denominator degree of freedom.

Keywords: GLMM; cluster-randomized trial; covariate adjustment; small number of clusters.

FIGURE A1

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 1 when the outcome is binary in the third simulation.

FIGURE A2

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 1 when the outcome is a count in the third simulation.

FIGURE 1

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 1 when the outcome is binary in the first simulation.

FIGURE 2

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 1 when the outcome is a count in the first simulation.

FIGURE 3

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 4 when the outcome is binary in the second simulation.

FIGURE 4

Type I error rates of tests on the treatment effect for data-generating model D and fitted model 4 when the outcome is a count in the second simulation.