Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Jul 24:3:82.
doi: 10.1186/2046-4053-3-82.

A checklist designed to aid consistency and reproducibility of GRADE assessments: development and pilot validation

Nick Meader  1 , Kristel King, Alexis Llewellyn, Gill Norman, Jennifer Brown, Mark Rodgers, Thirimon Moe-Byrne, Julian Pt Higgins, Amanda Sowden, Gavin Stewart
Affiliations

A checklist designed to aid consistency and reproducibility of GRADE assessments: development and pilot validation

Nick Meader et al. Syst Rev. .

Abstract

Background: The grading of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in health technology assessment and guideline development organisations throughout the world. GRADE provides a transparent approach to reaching judgements about the quality of evidence on the effects of a health care intervention, but is complex and therefore challenging to apply in a consistent manner.

Methods: We developed a checklist to guide the researcher to extract the data required to make a GRADE assessment. We applied the checklist to 29 meta-analyses of randomised controlled trials on the effectiveness of health care interventions. Two reviewers used the checklist for each paper and used these data to rate the quality of evidence for a particular outcome.

Results: For most (70%) checklist items, there was good agreement between reviewers. The main problems were for items relating to indirectness where considerable judgement is required.

Conclusions: There was consistent agreement between reviewers on most items in the checklist. The use of this checklist may be an aid to improving the consistency and reproducibility of GRADE assessments, particularly for inexperienced users or in rapid reviews without the resources to conduct assessments by two researchers independently.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Logic model for developing the checklist based on GRADE criteria. A logic model illustrating how the checklist items relate to the risk of bias, inconsistency, indirectness, imprecision and publication bias domains in GRADE.
See this image and copyright information in PMC

References

    1. Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knotterus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2011;64:380–382. - PubMed
    1. Guyatt GH, Oxman AD, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE Working Group. Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–926. - PMC - PubMed
    1. Guyatt G, Oxman AD, Akl E, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, Debeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ. GRADE guidelines 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–394. - PubMed
    1. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, Alderson P, Glasziou P, Falck-Ytter Y, Schunemann HJ. GRADE guidelines 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64:395–400. - PubMed
    1. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines 3: rating the quality of evidence - introduction. J Clin Epidemiol. 2011;64:401–406. - PubMed

Publication types

LinkOut - more resources