Reviews Pooling Heterogeneous, Low-Evidence, High-Bias Data Result in Incorrect Conclusions: But Heterogeneity is an Opportunity to Explore

Abstract

Systematic Review submissions to our journal commonly pool heterogeneous studies of low levels of evidence and a high risk of bias. Pooling, or quantitative synthesis, of such study data regularly results in incorrect conclusions. We reject these submissions without peer-review (desk rejection), and typically invite authors to submit a new, subjective synthesis without pooling and to report ranges of the results of included studies rather than pooled values. Generally, quantitative synthesis, or meta-analysis, should restrict included studies to randomized controlled trials. However, systematic review with exploration of heterogeneity can result in valuable information toward determining strengths and deficiencies of current literature, and thus guide future research.