Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2017 Sep 1;196(5):621-627.
doi: 10.1164/rccm.201705-0926OC.

A Comparative Analysis of Pulmonary and Critical Care Medicine Guideline Development Methodologies

Noah C Schoenberg  1 , Alan F Barker  2 , John Bernardo  1 , Robin R Deterding  3 , Jerrold J Ellner  1 , Dean R Hess  4 , Neil R MacIntyre  5 , Fernando J Martinez  6 , Kevin C Wilson  1   7
Affiliations
Comparative Study

A Comparative Analysis of Pulmonary and Critical Care Medicine Guideline Development Methodologies

Noah C Schoenberg et al. Am J Respir Crit Care Med. .

Abstract

Rationale: The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable.

Objectives: To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus.

Methods: The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured.

Measurements and main results: When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002).

Conclusions: We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOM-compliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.

Keywords: Grading of Recommendations, Assessment, Development, and Evaluation approach; Institute of Medicine standards for trustworthy guidelines; clinical practice guidelines; methodology.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
The Convergence of Opinion on Recommendations and Evidence process. IOM = Institute of Medicine.
Figure 2.
Figure 2.
Proposed Convergence of Opinion on Recommendations and Evidence (CORE) process utilization algorithm. IOM = Institute of Medicine.
See this image and copyright information in PMC

Comment in

  • Developing Clinical Guidelines: 99% Faster Is Not Enough.
    Krishnan JA. Krishnan JA. Am J Respir Crit Care Med. 2017 Sep 1;196(5):543-544. doi: 10.1164/rccm.201707-1542ED. Am J Respir Crit Care Med. 2017. PMID: 28759263 No abstract available.
  • Developing Clinical Guidelines.
    Morice AH, Hart S, Millqvist E. Morice AH, et al. Am J Respir Crit Care Med. 2018 Mar 15;197(6):837. doi: 10.1164/rccm.201708-1754LE. Am J Respir Crit Care Med. 2018. PMID: 29087207 No abstract available.
  • Reply to Morice et al.: Developing Clinical Guidelines.
    Schoenberg NC, Wilson KC. Schoenberg NC, et al. Am J Respir Crit Care Med. 2018 Mar 15;197(6):837-838. doi: 10.1164/rccm.201710-2039LE. Am J Respir Crit Care Med. 2018. PMID: 29087210 Free PMC article. No abstract available.
  • Have We Not Learned from Past Mistakes?
    Shah R, Morgan RL, Falck-Ytter Y, Mustafa RA. Shah R, et al. Am J Respir Crit Care Med. 2018 Jun 1;197(11):1499-1500. doi: 10.1164/rccm.201711-2200LE. Am J Respir Crit Care Med. 2018. PMID: 29345959 No abstract available.
  • No Room for Error in Medicine-A Case of Déjà Vu.
    Schünemann HJ, Brożek JL. Schünemann HJ, et al. Am J Respir Crit Care Med. 2018 Jun 1;197(11):1501-1502. doi: 10.1164/rccm.201710-2076LE. Am J Respir Crit Care Med. 2018. PMID: 29345966 No abstract available.
  • Reply to Dahm et al., to Shah et al., and to Schünemann and Brożek.
    Schoenberg NC, Wilson KC; all authors. Schoenberg NC, et al. Am J Respir Crit Care Med. 2018 Jun 1;197(11):1502-1503. doi: 10.1164/rccm.201712-2433LE. Am J Respir Crit Care Med. 2018. PMID: 29345969 No abstract available.
  • A Blast from the Past-Back to the 1970s.
    Dahm P, Sultan S, Murad MH. Dahm P, et al. Am J Respir Crit Care Med. 2018 Jun 1;197(11):1500-1501. doi: 10.1164/rccm.201711-2186LE. Am J Respir Crit Care Med. 2018. PMID: 29345971 No abstract available.

References

    1. Institute of Medicine. Washington, DC: Institute of Medicine; 2011. Clinical practice guidelines we can trust: standards for developing trustworthy clinical practice guidelines (CPGs) [accessed 2016 Oct 10]. Available from: http://www.nationalacademies.org/hmd/Reports/2011/Clinical-Practice-Guid....
    1. Qaseem A. Rockville, MD: National Guideline Clearinghouse, Agency for Healthcare Research and Quality; 2013. A perspective on the Guidelines International Network and the Institute of Medicine’s Proposed Standards for Guideline Development. [accessed 2017 Jul 31]. Available from: https://www.guideline.gov/expert/expert-commentary/43913/aperspective-on....
    1. Schünemann HJ, Jaeschke R, Cook DJ, Bria WF, El-Solh AA, Ernst A, Fahy BF, Gould MK, Horan KL, Krishnan JA, et al. ATS Documents Development and Implementation Committee. An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. 2006;174:605–614. - PubMed
    1. Guyatt G, Akl EA, Oxman A, Wilson K, Puhan MA, Wilt T, Gutterman D, Woodhead M, Antman EM, Schünemann HJ ATS/ERS Ad Hoc Committee on Integrating and Coordinating Efforts in COPD Guideline Development. Synthesis, grading, and presentation of evidence in guidelines: article 7 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. 2012;9:256–261. - PubMed
    1. Jacobs AK, Kushner FG, Ettinger SM, Guyton RA, Anderson JL, Ohman EM, Albert NM, Antman EM, Arnett DK, Bertolet M, et al. ACCF/AHA clinical practice guideline methodology summit report: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. 2013;127:268–310. - PubMed

Publication types