Estimating required information size by quantifying diversity in random-effects model meta-analyses

Abstract

Background: There is increasing awareness that meta-analyses require a sufficiently large information size to detect or reject an anticipated intervention effect. The required information size in a meta-analysis may be calculated from an anticipated a priori intervention effect or from an intervention effect suggested by trials with low-risk of bias.

Methods: Information size calculations need to consider the total model variance in a meta-analysis to control type I and type II errors. Here, we derive an adjusting factor for the required information size under any random-effects model meta-analysis.

Results: We devise a measure of diversity (D2) in a meta-analysis, which is the relative variance reduction when the meta-analysis model is changed from a random-effects into a fixed-effect model. D2 is the percentage that the between-trial variability constitutes of the sum of the between-trial variability and a sampling error estimate considering the required information size. D2 is different from the intuitively obvious adjusting factor based on the common quantification of heterogeneity, the inconsistency (I2), which may underestimate the required information size. Thus, D2 and I2 are compared and interpreted using several simulations and clinical examples. In addition we show mathematically that diversity is equal to or greater than inconsistency, that is D2 >or= I2, for all meta-analyses.

Conclusion: We conclude that D2 seems a better alternative than I2 to consider model variation in any random-effects meta-analysis despite the choice of the between trial variance estimator that constitutes the model. Furthermore, D2 can readily adjust the required information size in any random-effects model meta-analysis.

Figure 1

Diversity (D²) compared to inconsistency (I²) in 10,000 simulations of meta-analyses with number of trials included k = 6. Odds ratio = 1.00 and proportion of events in control group PC = 0.30. Meta-analyses depicted as open circles. D² nears asymptotically to I² when heterogeneity nears 0% or 100%. Line of unity, D² = I² black line.

Figure 2

Diversity (D²) compared to inconsistency (I²) in 10,000 simulations of meta-analyses with number of trials included k = 20. Odds ratio = 0.70 and proportion of events in control group PC = 0.30. Meta-analyses depicted as open circles. D² nears asymptotically to I² when heterogeneity nears 0% or 100%. Line of unity, D² = I² black line.

Figure 3

Diversity (D²) in % compared to inconsistency (I²) in % in seven meta-analyses (see Table 1) depicted as open circles. The open circles indicate that D² is always equal to or larger than I². 100% heterogeneity is impossible and the upper right point is just to illustrate that D² nears asymptotically to I² when heterogeneity nears 100%. Line of unity, D² = I² is the dotted blue line.