GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

doi:10.1016/j.jclinepi.2018.01.013

. 2019 Jul:111:94-104.

doi: 10.1016/j.jclinepi.2018.01.013. Epub 2018 Feb 13.

GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

Yuan Zhang¹, Pablo Alonso-Coello², Gordon H Guyatt¹, Juan José Yepes-Nuñez¹, Elie A Akl³, Glen Hazlewood⁴, Hector Pardo-Hernandez⁵, Itziar Etxeandia-Ikobaltzeta¹, Amir Qaseem⁶, John W Williams Jr⁷, Peter Tugwell⁸, Signe Flottorp⁹, Yaping Chang¹, Yuqing Zhang¹, Reem A Mustafa¹⁰, María Ximena Rojas¹¹, Holger J Schünemann¹²

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada.
² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain. Electronic address: PAlonso@santpau.cat.
³ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Faculty of Medicine, American University of Beirut, Beirut, Lebanon.
⁴ Department of Medicine and Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.
⁵ Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain.
⁶ Department of Clinical Policy, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106, USA.
⁷ Center of Innovation for Health Services Research in Primary Care, Durham Veterans Affairs Medical Center and Duke University, Durham, NC 27701, USA.
⁸ Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁹ Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway; Department of Health Management and Health Economics, Institute of Health and Society, University of Oslo, Oslo, Norway.
¹⁰ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Division of Nephrology and Hypertension, University of Kansas Medical Center, Kansas City, KS, USA.
¹¹ Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Bogotá, Colombia.
¹² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.

PMID: 29452223
DOI: 10.1016/j.jclinepi.2018.01.013

GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

Yuan Zhang et al. J Clin Epidemiol. 2019 Jul.

. 2019 Jul:111:94-104.

doi: 10.1016/j.jclinepi.2018.01.013. Epub 2018 Feb 13.

Authors

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada.
² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain. Electronic address: PAlonso@santpau.cat.
³ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Faculty of Medicine, American University of Beirut, Beirut, Lebanon.
⁴ Department of Medicine and Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.
⁵ Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain.
⁶ Department of Clinical Policy, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106, USA.
⁷ Center of Innovation for Health Services Research in Primary Care, Durham Veterans Affairs Medical Center and Duke University, Durham, NC 27701, USA.
⁸ Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁹ Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway; Department of Health Management and Health Economics, Institute of Health and Society, University of Oslo, Oslo, Norway.
¹⁰ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Division of Nephrology and Hypertension, University of Kansas Medical Center, Kansas City, KS, USA.
¹¹ Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Bogotá, Colombia.
¹² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.

PMID: 29452223
DOI: 10.1016/j.jclinepi.2018.01.013

Abstract

Objectives: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group defines patient values and preferences as the relative importance patients place on the main health outcomes. We provide GRADE guidance for assessing the risk of bias and indirectness domains for certainty of evidence about the relative importance of outcomes.

Study design and setting: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted GRADE members and other stakeholders for feedback.

Results: This is the first of two articles. A body of evidence addressing the importance of outcomes starts at "high certainty"; concerns with risk of bias, indirectness, inconsistency, imprecision, and publication bias lead to downgrading to moderate, low, or very low certainty. We propose subdomains of risk of bias as selection of the study population, missing data, the type of measurement instrument, and confounding; we have developed items for each subdomain. The population, intervention, comparison, and outcome elements associated with the evidence determine the degree of indirectness.

Conclusion: This article provides guidance and examples for rating the risk of bias and indirectness for a body of evidence summarizing the importance of outcomes.

Keywords: GRADE; Importance of outcomes; Indirectness; Quality of evidence; Risk of bias; Value and preference.

PubMed Disclaimer

Cited by

Effects of Therapies Involving Plyometric-Jump Training on Physical Fitness of Youth with Cerebral Palsy: A Systematic Review with Meta-Analysis.
Garcia-Carrillo E, Ramirez-Campillo R, Izquierdo M, Elnaggar RK, Afonso J, Peñailillo L, Araneda R, Ebner-Karestinos D, Granacher U. Garcia-Carrillo E, et al. Sports (Basel). 2024 May 29;12(6):152. doi: 10.3390/sports12060152. Sports (Basel). 2024. PMID: 38921846 Free PMC article. Review.
Effects of Upper-Body Plyometric Training on Physical Fitness in Healthy Youth and Young Adult Participants: A Systematic Review with Meta-Analysis.
Garcia-Carrillo E, Ramirez-Campillo R, Thapa RK, Afonso J, Granacher U, Izquierdo M. Garcia-Carrillo E, et al. Sports Med Open. 2023 Oct 13;9(1):93. doi: 10.1186/s40798-023-00631-2. Sports Med Open. 2023. PMID: 37833510 Free PMC article.
Values and preferences towards medical cannabis among people living with chronic pain: a mixed-methods systematic review.
Zeng L, Lytvyn L, Wang X, Kithulegoda N, Agterberg S, Shergill Y, Esfahani MA, Heen AF, Agoritsas T, Guyatt GH, Busse JW. Zeng L, et al. BMJ Open. 2021 Sep 7;11(9):e050831. doi: 10.1136/bmjopen-2021-050831. BMJ Open. 2021. PMID: 34493521 Free PMC article.
Impact of Lower-Volume Training on Physical Fitness Adaptations in Team Sports Players: A Systematic Review and Meta-analysis.
Clemente FM, Ramirez-Campillo R, Moran J, Zmijewski P, Silva RM, Randers MB. Clemente FM, et al. Sports Med Open. 2025 Jan 20;11(1):3. doi: 10.1186/s40798-024-00808-3. Sports Med Open. 2025. PMID: 39832098 Free PMC article.
Topical antimicrobial treatment of mesh for the reduction of surgical site infections after hernia repair: a systematic review and meta-analysis.
Bontekoning N, Huizing NJ, Timmer AS, Groenen H, de Jonge SW, Boermeester MA. Bontekoning N, et al. Hernia. 2024 Jun;28(3):691-700. doi: 10.1007/s10029-024-02987-0. Epub 2024 May 9. Hernia. 2024. PMID: 38722398 Free PMC article.

See all "Cited by" articles

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

CIHR/Canada

LinkOut - more resources

Full Text Sources
- ClinicalKey
- Elsevier Science
Other Literature Sources
- scite Smart Citations

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

Affiliations

GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

Authors

Affiliations

Abstract

Similar articles

Cited by

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources