Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 May 1;3(5):e205535.
doi: 10.1001/jamanetworkopen.2020.5535.

Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

Melissa C Brouwers  1 , Karen Spithoff  2 , Kate Kerkvliet  2 , Pablo Alonso-Coello  3 , Jako Burgers  4 , Francoise Cluzeau  5 , Beatrice Férvers  6 , Ian Graham  7 , Jeremy Grimshaw  7 , Steven Hanna  2 , Monika Kastner  8 , Michelle Kho  9 , Amir Qaseem  10 , Sharon Straus  11 , Ivan D Florez  12
Affiliations

Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

Melissa C Brouwers et al. JAMA Netw Open. .

Abstract

Importance: Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined.

Objective: To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations.

Design, setting, and participants: A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users.

Exposures: Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey.

Main outcomes and measures: Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach α, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters.

Results: A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach α = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76).

Conclusions and relevance: This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Brouwers reported receiving grants from the Canadian Institute for Health Research during the conduct of the study. Mss Spithoff and Kerkvliet reported receiving grants from the Canadian Institute for Health Research during the conduct of the study. Dr Burgers reported serving as Trustee of the AGREE Research Trust from 2004 to 2014. No other disclosures were reported.

References

    1. Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM. Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003;139(6):493-498. doi:10.7326/0003-4819-139-6-200309160-00013 - DOI - PubMed
    1. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P; Board of Trustees of the Guidelines International Network . Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525-531. doi:10.7326/0003-4819-156-7-201204030-00009 - DOI - PubMed
    1. Institute of Medicine Clinical Practice Guidelines We Can Trust. National Academies Press; 2011. - PubMed
    1. AGREE Collaboration Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18-23. doi:10.1136/qhc.12.1.18 - DOI - PMC - PubMed
    1. Brouwers MC, Kho ME, Browman GP, et al. ; AGREE Next Steps Consortium . AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839-E842. doi:10.1503/cmaj.090449 - DOI - PMC - PubMed

Publication types