Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study

Abstract

Background: Systematic reviews and meta-analyses of binary outcomes are widespread in all areas of application. The odds ratio, in particular, is by far the most popular effect measure. However, the standard meta-analysis of odds ratios using a random-effects model has a number of potential problems. An attractive alternative approach for the meta-analysis of binary outcomes uses a class of generalized linear mixed models (GLMMs). GLMMs are believed to overcome the problems of the standard random-effects model because they use a correct binomial-normal likelihood. However, this belief is based on theoretical considerations, and no sufficient simulations have assessed the performance of GLMMs in meta-analysis. This gap may be due to the computational complexity of these models and the resulting considerable time requirements.

Methods: The present study is the first to provide extensive simulations on the performance of four GLMM methods (models with fixed and random study effects and two conditional methods) for meta-analysis of odds ratios in comparison to the standard random effects model.

Results: In our simulations, the hypergeometric-normal model provided less biased estimation of the heterogeneity variance than the standard random-effects meta-analysis using the restricted maximum likelihood (REML) estimation when the data were sparse, but the REML method performed similarly for the point estimation of the odds ratio, and better for the interval estimation.

Conclusions: It is difficult to recommend the use of GLMMs in the practice of meta-analysis. The problem of finding uniformly good methods of the meta-analysis for binary outcomes is still open.

Keywords: Generalized linear mixed-effects models; Hypergeometric-normal likelihood; Meta-analysis; Random effects; Transformation bias.

Fig. 1

Bias of τ² in the REM when p_i2=0.1, θ=0, 0≤τ²≤1 and n=50,100,250,1000. Estimation methods are: pluses - unconditional generalized linear mixed-effects model with fixed study effects (FIM), crosses - unconditional generalized linear mixed-effects model with random study effects (RIM), circles - a conditional generalized linear mixed-effects model with exact likelihood (NCHGN), triangles - a conditional generalized linear mixed-effects model with approximate likelihood (ABNM), rhombs - DerSimonian and Laird method (DL) and reverse triangles - restricted maximum likelihood method (REML). Light grey line at 0

Fig. 2

Bias of θ in the REM when p_i2=0.1, θ=0, 0≤τ²≤1 and n=50,100,250,1000. Estimation methods are: pluses - unconditional generalized linear mixed-effects model with fixed study effects (FIM), crosses - unconditional generalized linear mixed-effects model with random study effects (RIM), circles - a conditional generalized linear mixed-effects model with exact likelihood (NCHGN), triangles - a conditional generalized linear mixed-effects model with approximate likelihood (ABNM), rhombs - DerSimonian and Laird method (DL) and reverse triangles - restricted maximum likelihood method (REML). Light grey line at 0

Fig. 3

Estimated coverage of θ in the REM when p_i2=0.1, θ=0, 0≤τ²≤1 and n=50,100,250,1000. The coverages are given at the nominal 95% level. Estimation methods are: pluses - unconditional generalized linear mixed-effects model with fixed study effects (FIM), crosses - unconditional generalized linear mixed-effects model with random study effects (RIM), circles - a conditional generalized linear mixed-effects model with exact likelihood (NCHGN), triangles - a conditional generalized linear mixed-effects model with approximate likelihood (ABNM), rhombs - DerSimonian and Laird method (DL) and reverse triangles - restricted maximum likelihood method (REML). Light grey line at 0.95

Fig. 4

Bias of τ² in the REM when p_i2=0.4, θ=1, 0≤τ²≤1 and n=50,100,250,1000. Estimation methods are: pluses - unconditional generalized linear mixed-effects model with fixed study effects (FIM), crosses - unconditional generalized linear mixed-effects model with random study effects (RIM), circles - a conditional generalized linear mixed-effects model with exact likelihood (NCHGN), triangles - a conditional generalized linear mixed-effects model with approximate likelihood (ABNM), rhombs - DerSimonian and Laird method (DL) and reverse triangles - restricted maximum likelihood method (REML). Light grey line at 0.95

Fig. 5

Proportion of convergence in the conditional generalized linear mixed-effects model with exact likelihood. These proportions of convergence are for p_i2=0.1, p_i2=0.2, p_i2=0.4, θ=0, and 0≤τ²≤1 for sample sizes n=50,100,250,1000 in each arm

Fig. 6

Proportion of convergence in the conditional generalized linear mixed-effects model with exact likelihood. These proportions of convergence are for p_i2=0.1, p_i2=0.2, p_i2=0.4, θ=1, and 0≤τ²≤1 for sample sizes n=50,100,250,1000 in each arm