GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy

doi:10.1016/j.jclinepi.2019.12.020

. 2020 Jun:122:129-141.

doi: 10.1016/j.jclinepi.2019.12.020. Epub 2020 Feb 12.

GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy

Holger J Schünemann¹, Reem A Mustafa², Jan Brozek³, Karen R Steingart⁴, Mariska Leeflang⁵, Mohammad Hassan Murad⁶, Patrick Bossuyt⁵, Paul Glasziou⁷, Roman Jaeschke³, Stefan Lange⁸, Joerg Meerpohl⁹, Miranda Langendam⁵, Monica Hultcrantz¹⁰, Gunn E Vist¹¹, Elie A Akl¹², Mark Helfand¹³, Nancy Santesso³, Lotty Hooft¹⁴, Rob Scholten¹⁴, Måns Rosen¹⁰, Anne Rutjes¹⁵, Mark Crowther³, Paola Muti¹⁶, Heike Raatz¹⁷, Mohammed T Ansari¹⁸, John Williams¹⁹, Regina Kunz²⁰, Jeff Harris²¹, Ingrid Arévalo Rodriguez²², Mikashmi Kohli²³, Gordon H Guyatt²⁴; GRADE Working Group

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada. Electronic address: schuneh@mcmaster.ca.
² Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
³ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada.
⁴ Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK.
⁵ Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, Room J1b-214, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.
⁶ Division of Preventive Medicine, Mayo Clinic, 200 1st Street SW, Rochester, MN 55902, USA.
⁷ CREBP, Faculty Health Science & Medicine, Bond University, Gold Coast QLD 4229, Australia.
⁸ Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, Institute for Quality and Efficiency in Health Care (IQWiG), Im Mediapark 8, 50670 Köln, Germany Cologne, Germany.
⁹ Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany.
¹⁰ Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden.
¹¹ Norwegian Knowledge Centre for the Health Services, PO Box 7004, St Olavs Plass, 0130 Oslo, Norway.
¹² Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut 1107 2020, Lebanon.
¹³ Oregon Evidence-based Practice Center, Oregon Health & Science University, Portland VA Medical Center, Portland, OR, USA.
¹⁴ Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, 3508 GA Utrecht, The Netherlands.
¹⁵ Clinical Trial Unit (CTU) Bern, Institute of Primary Health Care, University of Bern, Bern, Switzerland.
¹⁶ Department of Oncology, McMaster University, 711 Concession Street, Hamilton, Ontario L8V1C3, Canada.
¹⁷ University of Basel, Klingelbergstrasse 61, CH-4056 Basel, Switzerland; Kleijnen Systematic Reviews Ltd, 6 Escrick Business Park, Escrick, York YO19 6FD, UK.
¹⁸ School of Epidemiology and Public Health, Faculty of Medicine, Ottawa, Canada.
¹⁹ Duke University Medical Center and Durham Veterans Affairs Center for Health Services Research in Primary Care Durham, NC 27705, USA.
²⁰ Basel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, Basel 4031, Switzerland.
²¹ Harris Associates, 386 Richardson Way, Mill Valley, CA 94941, USA.
²² Clinical Biostatistics Unit, Hospital Universitario Ramón y Cajal, IRYCIS, CIBER of Epidemiology and Public Health, Madrid, Spain; Centro de investigación en Salud Pública y Epidemiología Clínica, Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador.
²³ Department of Epidemiology, Biostatistics and Occupational Health, McGill University, 1650 Cedar Avenue, Montreal, Quebec H3G 1A4, Canada.
²⁴ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA.

PMID: 32060007
DOI: 10.1016/j.jclinepi.2019.12.020

GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy

Holger J Schünemann et al. J Clin Epidemiol. 2020 Jun.

. 2020 Jun:122:129-141.

doi: 10.1016/j.jclinepi.2019.12.020. Epub 2020 Feb 12.

Authors

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada. Electronic address: schuneh@mcmaster.ca.
² Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA.
³ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada.
⁴ Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK.
⁵ Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, Room J1b-214, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.
⁶ Division of Preventive Medicine, Mayo Clinic, 200 1st Street SW, Rochester, MN 55902, USA.
⁷ CREBP, Faculty Health Science & Medicine, Bond University, Gold Coast QLD 4229, Australia.
⁸ Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, Institute for Quality and Efficiency in Health Care (IQWiG), Im Mediapark 8, 50670 Köln, Germany Cologne, Germany.
⁹ Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany.
¹⁰ Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden.
¹¹ Norwegian Knowledge Centre for the Health Services, PO Box 7004, St Olavs Plass, 0130 Oslo, Norway.
¹² Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut 1107 2020, Lebanon.
¹³ Oregon Evidence-based Practice Center, Oregon Health & Science University, Portland VA Medical Center, Portland, OR, USA.
¹⁴ Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, 3508 GA Utrecht, The Netherlands.
¹⁵ Clinical Trial Unit (CTU) Bern, Institute of Primary Health Care, University of Bern, Bern, Switzerland.
¹⁶ Department of Oncology, McMaster University, 711 Concession Street, Hamilton, Ontario L8V1C3, Canada.
¹⁷ University of Basel, Klingelbergstrasse 61, CH-4056 Basel, Switzerland; Kleijnen Systematic Reviews Ltd, 6 Escrick Business Park, Escrick, York YO19 6FD, UK.
¹⁸ School of Epidemiology and Public Health, Faculty of Medicine, Ottawa, Canada.
¹⁹ Duke University Medical Center and Durham Veterans Affairs Center for Health Services Research in Primary Care Durham, NC 27705, USA.
²⁰ Basel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, Basel 4031, Switzerland.
²¹ Harris Associates, 386 Richardson Way, Mill Valley, CA 94941, USA.
²² Clinical Biostatistics Unit, Hospital Universitario Ramón y Cajal, IRYCIS, CIBER of Epidemiology and Public Health, Madrid, Spain; Centro de investigación en Salud Pública y Epidemiología Clínica, Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador.
²³ Department of Epidemiology, Biostatistics and Occupational Health, McGill University, 1650 Cedar Avenue, Montreal, Quebec H3G 1A4, Canada.
²⁴ Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA.

PMID: 32060007
DOI: 10.1016/j.jclinepi.2019.12.020

Abstract

Objectives: This article provides updated GRADE guidance about how authors of systematic reviews and health technology assessments and guideline developers can assess the results and the certainty of evidence (also known as quality of the evidence or confidence in the estimates) of a body of evidence addressing test accuracy (TA).

Study design and setting: We present an overview of the GRADE approach and guidance for rating certainty in TA in clinical and public health and review the presentation of results of a body of evidence regarding tests. Part 1 of the two parts in this 21st guidance article about how to apply GRADE focuses on understanding study design issues in test accuracy, provide an overview of the domains, and describe risk of bias and indirectness specifically.

Results: Supplemented by practical examples, we describe how raters of the evidence using GRADE can evaluate study designs focusing on tests and how they apply the GRADE domains risk of bias and indirectness to a body of evidence of TA studies.

Conclusion: Rating the certainty of a body of evidence using GRADE in Cochrane and other reviews and World Health Organization and other guidelines dealing with in TA studies helped refining our approach. The resulting guidance will help applying GRADE successfully for questions and recommendations focusing on tests.

Keywords: Certainty of evidence; Diagnosis; Diagnostic accuracy; GRADE; Test accuracy; Tests.

PubMed Disclaimer

Cited by

Electrical conductance for the detection of dental caries.
Macey R, Walsh T, Riley P, Glenny AM, Worthington HV, Clarkson JE, Ricketts D. Macey R, et al. Cochrane Database Syst Rev. 2021 Mar 16;3(3):CD014547. doi: 10.1002/14651858.CD014547. Cochrane Database Syst Rev. 2021. PMID: 33724442 Free PMC article.
AGA Institute Rapid Review and Recommendations on the Role of Pre-Procedure SARS-CoV-2 Testing and Endoscopy.
Sultan S, Siddique SM, Altayar O, Caliendo AM, Davitkov P, Feuerstein JD, Francis D, Inadomi JM, Lim JK, Falck-Ytter Y, Mustafa RA; American Gastroenterological Association. Electronic address: ewilson@gastro.org. Sultan S, et al. Gastroenterology. 2020 Nov;159(5):1935-1948.e5. doi: 10.1053/j.gastro.2020.07.043. Epub 2020 Jul 28. Gastroenterology. 2020. PMID: 32735862 Free PMC article. Review. No abstract available.
Relationship between Bioelectrical Impedance Phase Angle and Upper and Lower Limb Muscle Strength in Athletes from Several Sports: A Systematic Review with Meta-Analysis.
Cirillo E, Pompeo A, Cirillo FT, Vilaça-Alves J, Costa P, Ramirez-Campillo R, Dourado AC, Afonso J, Casanova F. Cirillo E, et al. Sports (Basel). 2023 May 18;11(5):107. doi: 10.3390/sports11050107. Sports (Basel). 2023. PMID: 37234063 Free PMC article. Review.
Guidelines for the Neurocritical Care Management of Aneurysmal Subarachnoid Hemorrhage.
Treggiari MM, Rabinstein AA, Busl KM, Caylor MM, Citerio G, Deem S, Diringer M, Fox E, Livesay S, Sheth KN, Suarez JI, Tjoumakaris S. Treggiari MM, et al. Neurocrit Care. 2023 Aug;39(1):1-28. doi: 10.1007/s12028-023-01713-5. Epub 2023 May 18. Neurocrit Care. 2023. PMID: 37202712
Screening for the prevention and early detection of cervical cancer: protocol for systematic reviews to inform Canadian recommendations.
Gates A, Pillay J, Reynolds D, Stirling R, Traversy G, Korownyk C, Moore A, Thériault G, Thombs BD, Little J, Popadiuk C, van Niekerk D, Keto-Lambert D, Vandermeer B, Hartling L. Gates A, et al. Syst Rev. 2021 Jan 2;10(1):2. doi: 10.1186/s13643-020-01538-9. Syst Rev. 2021. PMID: 33388083 Free PMC article.

See all "Cited by" articles

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
- ClinicalKey
- Elsevier Science

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy

Affiliations

GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy

Authors

Affiliations

Abstract

Similar articles

Cited by

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources

Abstract

Similar articles

Cited by

Publication types

MeSH terms

Related information

LinkOut - more resources

Full Text Sources