High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews

Abstract

Objectives: To appraise the quality of evidence in systematic reviews (SRs) within the Cochrane Database of Systematic Reviews (CDSRs) across diverse topics and to explore the relationship between the strength of evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) and the probability that authors would interpret that an intervention may be of value.

Study design and setting: We evaluated the SRs published on the CDSR from January 1, 2013, to June 30, 2014. Two authors identified relevant SRs by independent searching of the Cochrane register. We further focused on SRs that incorporated tables with GRADE [summary of findings (SoF)]. Data were extracted independently by two authors. The quality of the evidence for the first listed primary outcome in SoF tables in each review and reasons for upgrade or downgrade were recorded.

Results: Overall, 1,394 SRs were identified. Of these, 608 (43.6%) incorporated GRADE. Within these reviews, only 13.5% (n = 82) reported a high quality and 30.8% (n = 187) a moderate quality of evidence for the first listed primary outcome, whereas 31.7% (n = 193) had low level and 24% (n = 146) had very low level of evidence. High quality of evidence was more common in updated compared to new reviews and in pharmacologic than other types of interventions. Even when all outcomes listed in the SoFs were considered, only 116/608 (19.1%) of SRs had at least one outcome with high quality of evidence. Overall, only 4.1% (25/608) of SRs incorporating GRADE in SoF tables had high quality of evidence, allied both to significant results and a favorable interpretation of the intervention by the reviewers.

Conclusion: Evidence of high quality is uncommon for medical and health-related interventions assessed with GRADE within the CDSR, and favorable evidence of high quality is even more uncommon.

Keywords: GRADE; Quality evidence; Randomized controlled trial; Systematic review; Treatment recommendation; ​Cochrane.