Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep:161:39-45.
doi: 10.1016/j.jclinepi.2023.06.015. Epub 2023 Jun 24.

Risk-of-bias assessment using Cochrane's revised tool for randomized trials (RoB 2) was useful but challenging and resource-intensive: observations from a systematic review

Thomas Frederick Crocker  1 , Natalie Lam  2 , Magda Jordão  2 , Caroline Brundle  2 , Matthew Prescott  2 , Anne Forster  2 , Joie Ensor  3 , John Gladman  4 , Andrew Clegg  2
Affiliations
Free article

Risk-of-bias assessment using Cochrane's revised tool for randomized trials (RoB 2) was useful but challenging and resource-intensive: observations from a systematic review

Thomas Frederick Crocker et al. J Clin Epidemiol. 2023 Sep.
Free article

Abstract

Objectives: To report our experience using version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2).

Study design and setting: Two reviewers independently applied RoB 2 to results of interest in a large systematic review of complex interventions and reached consensus. We recorded the time taken, and noted and discussed our difficulties using the tool, and the resolutions we adopted. We explored the time taken with regression analysis and summarized our experience of implementing the tool.

Results: We assessed risk of bias in 860 results of interest in 113 studies. Staff resource averaged 358 minutes per study (SD 183). Number of results (β = 22) and reports (β = 14) per study and experience of the team (β = -6) significantly affected assessment time. To implement the tool consistently, we developed cut points for missingness and considerations of balance regarding missingness, assumed some concerns with intervention deviations unless otherwise prevented or investigated, some concerns with measurements from unblinded self-reporting participants, and judged low risk of selection for certain dichotomous outcomes despite the absence of an analysis plan.

Conclusion: The RoB 2 tool and guidance are useful but resource-intensive and challenging to implement. Critical appraisal tools and reporting guidelines should detail risk of bias implementation. Improved guidance focusing on implementation could assist reviewers.

Keywords: Certainty assessment; Process duration; Research methods; Risk of bias; RoB2; Systematic reviews.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest Andrew Clegg declares funding through the NIHR HTA programme, NIHR Programme Grants for Applied Research, NIHR HS&DR programme, NIHR Applied Research Collaboration Yorkshire & Humber, and Health Data Research UK; Anne Forster declares NIHR Senior Investigator Award 2017-present, NIHR Programme Grant 10% of salary, NIHR HS&DR grant 8% of salary, HTA grant 5% of salary, National Institute for Health (USA) payment for panel membership 2021, 2022, participation in Programme Steering Committees for NIHR 202,339 Improving the lives of stroke survivors with data, and NIHR202020 Research Title Personalised Exercise-Rehabilitation FOR people with Multiple long-term conditions (multimorbidity)-The PERFORM trial, University of Leeds Governor representative on the Governors Board of Bradford Teaching Hospitals NHS Foundation Trust, member of HSDR Researcher-Led panel, member of NIHR Doctoral Fellowship Panel member of Policy Research Unit assessment panel. Other authors declare no potential conflicts of interest.

Publication types

LinkOut - more resources