High and unclear risk of bias assessments are predominant in diagnostic accuracy studies included in Cochrane reviews

Abstract

Objectives: The validity of outcomes of systematic reviews is highly dependent on the extent of bias in the included primary studies. This study reports the risk of bias (ROB) of primary studies included in systematic reviews of diagnostic accuracy.

Study design and setting: All systematic reviews of diagnostic accuracy studies published in the Cochrane database in 2015 and 2016 that used the Quality Assessment of Diagnostic Accuracy Studies-2 ROB tool and reported results with the ROB figure were eligible. The primary outcome was the prevalence of "high" or "unclear" ROB scores for the four Quality Assessment of Diagnostic Accuracy Studies-2 domains: "patient selection," "index test," "reference standard," and "flow and timing".

Results: Of 46 eligible reviews, 35 fulfilled the inclusion criteria. A total of 1045 primary studies with 4133 bias assessments were identified. Of those, 56% (2319/4133) were assessed to be at "high" or "unclear" ROB and 44% (1814/4133) at low ROB. For all domains except "flow and timing," most outcomes were scored as "high" or "unclear" ROB. A total of 47 (47/1045; 4.5%, 3.4 to 5.9%) primary studies were scored at low ROB for all domains. Older article age was significantly associated with likelihood of "high" or "unclear" ROB (odds ratio: 1.02; 95% confidence interval: 1.01 to 1.03; P < 0.001).

Conclusion: Systematic reviews of diagnostic accuracy are based on studies with a majority of "high" or "unclear" bias assessments. The age of the articles explained only a small part of the variability of the score assessments, therefore not justifying an a priori exclusion of older articles in systematic reviews. There is an urgent need to improve the quality of design, conduct, and reporting of diagnostic accuracy studies.

Keywords: Diagnostic accuracy; Reproducibility; Risk of bias; Study design; Systematic reviews.