Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2021 Nov 25:375:e066045.
doi: 10.1136/bmj-2021-066045.

Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis

Liang Yao  1 , Muhammad Muneeb Ahmed  2 , Gordon H Guyatt  1   3 , Peijing Yan  4 , Xu Hui  5 , Qi Wang  1 , Kehu Yang  5 , Jinhui Tian  5 , Benjamin Djulbegovic  6
Affiliations
Comparative Study

Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis

Liang Yao et al. BMJ. .

Abstract

Objective: To investigate whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelines.

Design: Empirical analysis.

Data source: Guidelines developed by the American College of Cardiology and the American Heart Association (ACC/AHA) and the American Society of Clinical Oncology (ASCO) up to 27 March 2021.

Study selection: Recommendations were clearly categorised as consensus or evidence based, were separated from the remainder of the text, and included both the quality of evidence and the strength of the recommendations.

Data extraction: Paired authors independently extracted the recommendation characteristics, including type of recommendation (consensus or evidence based), grading system used for developing recommendations, strength of the recommendation, and quality of evidence. The study team also calculated the number of discordant recommendations (strong recommendations with low quality evidence) and inappropriate discordant recommendations (those that did not meet grading of recommendations assessment, development, and evaluation criteria of appropriateness).

Results: The study included 12 ACC/AHA guidelines that generated 1434 recommendations and 69 ASCO guidelines that generated 1094 recommendations. Of the 504 ACC/AHA recommendations based on low quality evidence, 200 (40%) proved to be consensus based versus 304 (60%) evidence based; of the 404 ASCO recommendations based on low quality evidence, 292 (72%) were consensus based versus 112 (28%) that were evidence based. In both ACC/AHA and ASCO guidelines, the consensus approach yielded more discordant recommendations (ACC/AHA: odds ratio 2.1, 95% confidence interval 1.5 to 3.1; ASCO: 2.9, 1.1 to 7.8) and inappropriate discordant recommendations (ACC/AHA: 2.6, 1.7 to 3.7; ASCO: 5.1, 1.6 to 16.0) than the evidence based approach.

Conclusion: Consensus based guidelines produce more recommendations violating the evidence based medicine principles than evidence based guidelines. Ensuring appropriate alignment of quality of evidence with the strength of recommendations is key to the development of "trustworthy" guidelines.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from supported by the Agency for Healthcare Research and Quality for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Figures

Fig 1
Fig 1
The proportion of discordant recommendations and inappropriate discordant recommendations in consensus versus evidence based methods of guidelines development. The odds ratio (95% confidence interval) estimates were generated from the multilevel model (appendix 4). Odds ratio >1 indicates that guidelines developed by consensus based methods generate more discordant or inappropriate discordant recommendations than the guidelines that employ evidence based approaches. ACC/AHA=American College of Cardiology and the American Heart Association; ASCO=American Society of Clinical Oncology
See this image and copyright information in PMC

References

    1. Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines . In: Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, eds. Clinical Practice Guidelines We Can Trust. US National Academies Press, 2011. - PubMed
    1. Cook TM, El-Boghdadly K, McGuire B, McNarry AF, Patel A, Higgs A. Consensus guidelines for managing the airway in patients with COVID-19: Guidelines from the Difficult Airway Society, the Association of Anaesthetists the Intensive Care Society, the Faculty of Intensive Care Medicine and the Royal College of Anaesthetists. Anaesthesia 2020;75:785-99. 10.1111/anae.15054 - DOI - PMC - PubMed
    1. Lo STH, Yong AS, Sinhal A, et al. Interventional council of CSANZ and COVID-19 Interventional cardiology working group . Consensus guidelines for interventional cardiology services delivery during covid-19 pandemic in Australia and new Zealand. Heart Lung Circ 2020;29:e69-77. 10.1016/j.hlc.2020.04.002 - DOI - PMC - PubMed
    1. Hopkins C, Alanin M, Philpott C, et al. . Management of new onset loss of sense of smell during the COVID-19 pandemic - BRS Consensus Guidelines. Clin Otolaryngol 2021;46:16-22. 10.1111/coa.13636 - DOI - PMC - PubMed
    1. Ye Z, Rochwerg B, Wang Y, et al. . Treatment of patients with nonsevere and severe coronavirus disease 2019: an evidence-based guideline. CMAJ 2020;192:E536-45. 10.1503/cmaj.200648 - DOI - PMC - PubMed

Publication types