Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Dec;34(12):808-813.
doi: 10.1097/EJA.0000000000000691.

High quality of evidence is uncommon in Cochrane systematic reviews in Anaesthesia, Critical Care and Emergency Medicine

Aaron Conway  1 , Zachary Conway, Kathleen Soalheira, Joanna Sutherland
Affiliations

High quality of evidence is uncommon in Cochrane systematic reviews in Anaesthesia, Critical Care and Emergency Medicine

Aaron Conway et al. Eur J Anaesthesiol. 2017 Dec.

Abstract

Background: The association between the quality of evidence in systematic reviews and authors' conclusions regarding the effectiveness of interventions relevant to anaesthesia has not been examined.

Objective: The objectives of this study were: to determine the proportion of systematic reviews in which the authors made a conclusive statement about the effect of an intervention; to describe the quality of evidence derived from outcomes in reviews that used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) working group system for grading the quality of evidence; and to identify review characteristics associated with conclusiveness.

Design: Cross-sectional analysis of Cochrane systematic reviews from the Anaesthesia, Critical Care and Emergency Review Group was undertaken.

Data sources: The Cochrane webpage was used to identify reviews for inclusion (http://.ace.cochrane.org/).

Eligibility criteria: New and updated versions of systematic reviews published up to 17 September 2015 were eligible. Protocols for systematic reviews were excluded.

Results: A total of 159 reviews were included. GRADE was used in 103 reviews (65%). Of these, high-level evidence for the primary outcome was identified in 11 reviews (10%). The main reasons that quality of evidence for the primary outcome was downgraded were risk of bias (n = 44; 43%) and imprecision (n = 36; 35%). Authors of 47% (n = 75) of the total number of reviews made conclusive statements about the effects of interventions. Independent predictors of conclusiveness in the subgroup of reviews with GRADE assessments were quality of evidence for the primary outcome (odds ratio 2.03; 95% confidence interval: [1.18 to 3.52] and an increasing number of studies included in reviews (OR 1.05; 95% CI: [1.01 to 1.09]).

Conclusion: It was common for conclusive statements to be made about the effects of interventions despite evidence for the primary outcome being rated less than high quality. Improving methodological quality of trials would have the greatest impact on improving the quality of evidence.

PubMed Disclaimer

Comment in

References

    1. Fleming PS, Koletsi D, Ioannidis JP, et al. High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews. J Clin Epidemiol 2016; 78:34–42. - PubMed
    1. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336:924–926. - PMC - PubMed
    1. Higgins JP, Green S. The Cochrane Collaboration. Wiley-Blackwell, Cochrane handbook for systematic reviews of interventions. Hoboken: 2008.
    1. Punjasawadwong Y, Phongchiewboon A, Bunchungmongkol N. Bispectral index for improving anaesthetic delivery and postoperative recovery. Cochrane Database Syst Rev 2014; CD003843. - PMC - PubMed
    1. Pandis N, Fleming PS, Worthington H, et al. The quality of the evidence according to GRADE is predominantly low or very low in oral health systematic reviews. PloS One 2015; 10:e0131644. - PMC - PubMed

MeSH terms