Comparison of variance estimators for meta-analysis of instrumental variable estimates

Abstract

Background: Mendelian randomization studies perform instrumental variable (IV) analysis using genetic IVs. Results of individual Mendelian randomization studies can be pooled through meta-analysis. We explored how different variance estimators influence the meta-analysed IV estimate.

Methods: Two versions of the delta method (IV before or after pooling), four bootstrap estimators, a jack-knife estimator and a heteroscedasticity-consistent (HC) variance estimator were compared using simulation. Two types of meta-analyses were compared, a two-stage meta-analysis pooling results, and a one-stage meta-analysis pooling datasets.

Results: Using a two-stage meta-analysis, coverage of the point estimate using bootstrapped estimators deviated from nominal levels at weak instrument settings and/or outcome probabilities ≤ 0.10. The jack-knife estimator was the least biased resampling method, the HC estimator often failed at outcome probabilities ≤ 0.50 and overall the delta method estimators were the least biased. In the presence of between-study heterogeneity, the delta method before meta-analysis performed best. Using a one-stage meta-analysis all methods performed equally well and better than two-stage meta-analysis of greater or equal size.

Conclusions: In the presence of between-study heterogeneity, two-stage meta-analyses should preferentially use the delta method before meta-analysis. Weak instrument bias can be reduced by performing a one-stage meta-analysis.

Keywords: Epidemiology methods; Mendelian randomization analysis; statistics.

Figure 1.

Simulation results from scenarios I comparing different IV variance estimators. *Solid line with a square symbol, delta method followed by meta-analysis [DM1]; solid line with a circle symbol, basic bootstrap [BB]; solid line with triangle symbol, outcome-stratified bootstrap [OS]; solid line with a plus symbol, SNP-stratified bootstrap [SS]; solid line with a filled-out square symbol, double bootstrap [DB]; solid line with a filled-out circle symbol, jack-knife estimator [JK]; solid line with a filled-out triangle symbol, robust variance estimator [RB]; solid line with a rhombus (diamond) symbol, meta-analysis followed by delta method [DM2]. The DB y-value of 2.071 is not depicted for an MAF of 0.005 on the bottom left graph.

Figure 2.

Sensitivity analysis repeating simulation I comparing different IV variance estimators with an average of 60 000 subjects. *Solid line with a square symbol, delta method followed by meta-analysis [DM1]; solid line with a circle symbol, basic bootstrap [BB]; solid line with triangle symbol, outcome-stratified bootstrap [OS]; solid line with a plus symbol, SNP-stratified bootstrap [SS]; solid line with a filled-out square symbol, double bootstrap [DB]; solid line with a filled-out circle symbol, jack-knife estimator [JK]; solid line with a filled-out triangle symbol, robust variance estimator [RB]; solid line with a rhombus (diamond) symbol, meta-analysis followed by delta method [DM2].

Figure 3.

Sensitivity analysis repeating simulation I comparing different IV variance estimators using a one-stage meta-analysis design with an average of 20 000 subjects. *Solid line with a square symbol, delta method followed by meta-analysis [DM1]; solid line with a circle symbol, basic bootstrap [BB]; solid line with triangle symbol, outcome-stratified bootstrap [OS]; solid line with a plus symbol, SNP-stratified bootstrap [SS]; solid line with a filled-out square symbol, double bootstrap [DB]; solid line with a filled-out circle symbol, jack-knife estimator [JK]; solid line with a filled-out triangle symbol, robust variance estimator [RB]; solid line with a star symbol, bootstrapped percentile method. The BB y-value of -13.463 is not depicted for an MAF of 0.005 on the right graph.

Figure 4.

Bootstrap distributions for IV rs2965101 for the relation of LDL-C and CVD. *Solid grey lines indicate the non-parametric density (only presented in the second row), with dashed grey lines indicating the expected density given a normal distribution (not presented for the double bootstrap).