Use of quality assessment tools in systematic reviews was varied and inconsistent

Abstract

Objectives: To assess the use of quality assessment tools among a cross-section of systematic reviews (SRs) and to further evaluate whether quality was used as a parameter in the decision to include primary studies within subsequent meta-analysis.

Study design and setting: We searched PubMed for SRs (interventional, observational, and diagnostic) published in Core Clinical Journals between January 1 and March 31, 2014.

Results: Three hundred nine SRs were identified. Quality assessment was undertaken in 222 (71.8%) with isolated use of the Cochrane risk of bias tool (26.1%, n = 58) and the Newcastle-Ottawa Scale (15.3%, n = 34) most common. A threshold level of primary study quality for subsequent meta-analysis was used in 12.9% (40 of 309) of reviews. Overall, fifty-four combinations of quality assessment tools were identified with a similar preponderance of tools used among observational and interventional reviews. Multiple tools were used in 11.7% (n = 36) of SRs overall.

Conclusion: We found that quality assessment tools were used in a majority of SRs; however, a threshold level of quality for meta-analysis was stipulated in just 12.9% (n = 40). This cross-sectional analysis provides further evidence of the need for more active or intuitive editorial processes to enhance the reporting of SRs.

Keywords: Meta-analysis; Quality assessment; Review; Risk of bias; Systematic; Threshold.