Exclusion of studies with no events in both arms in meta-analysis impacted the conclusions

Abstract

Objectives: Classical meta-analyses routinely treated studies with no events in both arms noninformative and excluded them from analyses. This study assessed whether such studies contain information and have an influence on the conclusions of meta-analyses.

Study design and setting: We collected meta-analyses of binary outcomes with at least one study having no events in both arms from Cochrane systematic reviews (2003-2018). We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), P value, and width of 95% confidence interval (CI) were compared. A simulation study was conducted to examine the robustness of results.

Results: We identified 442 meta-analyses. In comparing paired meta-analyses that included studies with no events in both arms vs. those not, 8 (1.80%) resulted in different directions on OR; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on P value (55.66% increased and 44.12% decreased) and the width of 95% CI (50.68% inflated and 49.32% declined) when excluding studies with no events. Simulation study confirmed these findings.

Conclusion: Studies with no events in both arms are not necessarily noninformative. Excluding such studies may alter conclusions.

Keywords: Generalized linear mixed model; Meta-analysis of rare events; Statistical inference; Zero-events studies.