Detecting Heterogeneity of Intervention Effects Using Analysis and Meta-analysis of Differences in Variance Between Trial Arms

Abstract

Background: Randomized controlled trials (RCTs) with continuous outcomes usually only examine mean differences in response between trial arms. If the intervention has heterogeneous effects, then outcome variances will also differ between arms. Power of an individual trial to assess heterogeneity is lower than the power to detect the same size of main effect.

Methods: We describe several methods for assessing differences in variance in trial arms and apply them to a single trial with individual patient data and to meta-analyses using summary data. Where individual data are available, we use regression-based methods to examine the effects of covariates on variation. We present an additional method to meta-analyze differences in variances with summary data.

Results: In the single trial, there was agreement between methods, and the difference in variance was largely due to differences in prevalence of depression at baseline. In two meta-analyses, most individual trials did not show strong evidence of a difference in variance between arms, with wide confidence intervals. However, both meta-analyses showed evidence of greater variance in the control arm, and in one example, this was perhaps because mean outcome in the control arm was higher.

Conclusions: Using meta-analysis, we overcame low power of individual trials to examine differences in variance using meta-analysis. Evidence of differences in variance should be followed up to identify potential effect modifiers and explore other possible causes such as varying compliance.

FIGURE 1.

Forest plot of the ratio of variances and differences in variance analyses for the trials in the Richards et al. meta-analysis on computer-based psychological treatments for depression, results in eTable 6; http://links.lww.com/EDE/B835 (note we do not plot the results of the log ratio of SD analysis as trends are the same as the ratio of variances analysis). Please note that the studies named in the figure are those in the Richards et al. meta-analysis, and full information on these studies, including references, can be found in that article.

FIGURE 2.

Forest plot of the ratio of variances, differences in variance, and differences in covariance analyses of the trials in the Palmer et al. meta-analysis reporting the effect of statins versus placebo or no treatment on LDL cholesterol, results in eTable 7; http://links.lww.com/EDE/B835. We have not plotted the ratio of variances results for Aranda 1994 as the ratio of variances for this trial is on a much larger scale than the others (9.51 [95% CI 1.90, 47.49]); however, it is included in the overall analysis. Note we do not plot the results of the log ratios of SDs or log ratios of covariance analyses as trends were the same as the ratio of variances and differences in covariance analyses, respectively (eTable 7; http://links.lww.com/EDE/B835). Please note that the studies named in the figure are those in the Palmer et al. meta-analysis, and full information on these studies, including references, can be found in that paper.