Imputing intracluster correlation coefficients from a posterior predictive distribution is a feasible method of dealing with unit of analysis errors in a meta-analysis of cluster RCTs

Abstract

Background: Incorporating cluster randomized trials (CRTs) into meta-analyses is challenging because appropriate standard errors of study estimates accounting for clustering are not always reported. Systematic reviews of CRTs often use a single constant external estimate of the intraclass correlation coefficient (ICC) to adjust study estimate standard errors and facilitate meta-analyses; an approach that fails to account for possible variation of ICCs among studies and the imprecision with which they are estimated. Using a large systematic review of the effects of diabetes quality improvement interventions, we investigated whether we could better account for ICC variation and uncertainty in meta-analyzed effect estimates by imputing missing ICCs from a posterior predictive distribution constructed from a database of relevant ICCs.

Methods: We constructed a dataset of ICC estimates from applicable studies. For outcomes with two or more available ICC estimates, we constructed posterior predictive ICC distributions in a Bayesian framework. For a selected continuous outcome, glycosylated hemoglobin (HbA1c), we compared the impact of incorporating a single constant ICC versus imputing ICCs drawn from the posterior predictive distribution when estimating the effect of intervention components on post treatment mean in a case study of diabetes quality improvement trials.

Results: Using internal and external ICC estimates, we were able to construct a database of 59 ICCs for 12 of the 13 review outcomes (range 1-10 per outcome) and estimate the posterior predictive ICC distribution for 11 review outcomes. Synthesized results were not markedly changed by our approach for HbA1c.

Conclusion: Building posterior predictive distributions to impute missing ICCs is a feasible approach to facilitate principled meta-analyses of cluster randomized trials using prior data. Further work is needed to establish whether the application of these methods leads to improved review inferences for different reviews based on different factors (e.g., proportion of CRTs and CRTs with missing ICCs, different outcomes, variation and precision of ICCs).

Keywords: Cluster randomized trials; Complex interventions; Intraclass correlation coefficient; Meta-analysis; Unit of analysis errors; intracluster correlation coefficient.