Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 May;15(3):430-440.
doi: 10.1002/jrsm.1695. Epub 2024 Jan 23.

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study

Andrija Babić  1 , Ognjen Barcot  2 , Tomislav Visković  1 , Frano Šarić  3 , Aleksandar Kirkovski  4 , Ivana Barun  5 , Zvonimir Križanac  2 , Roshan Arjun Ananda  6 , Yuli Viviana Fuentes Barreiro  7 , Narges Malih  8 , Daiana Anne-Marie Dimcea  9 , Josipa Ordulj  10 , Ishanka Weerasekara  11 , Matteo Spezia  12 , Marija Franka Žuljević  13 , Jelena Šuto  14 , Luca Tancredi  15   16 , Anđela Pijuk  17 , Susanna Sammali  18   19 , Veronica Iascone  18 , Thilo von Groote  20 , Tina Poklepović Peričić  21 , Livia Puljak  22
Affiliations

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study

Andrija Babić et al. Res Synth Methods. 2024 May.

Abstract

Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.

Keywords: adequacy; cochrane risk of bias tool 2.0; improvement; inconsistency; non‐cochrane systematic reviews.

PubMed Disclaimer

References

REFERENCES

    1. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
    1. Sterne JAC, Savovic J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.
    1. Babic A, Pijuk A, Brazdilova L, et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med Res Methodol. 2019;19(1):77.
    1. Babic A, Tokalic R, Amilcar Silva Cunha J, et al. Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability. BMC Med Res Methodol. 2019;19(1):76.
    1. Babic A, Vuka I, Saric F, et al. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2020;119:57‐64.

LinkOut - more resources