New measures improved the reporting of heterogeneity in diagnostic test accuracy reviews: a metaepidemiological study

Abstract

Objectives: To describe the methods used to quantify heterogeneity and to propose alternative measures to improve reporting of heterogeneity in Cochrane diagnostic test accuracy (DTA) reviews.

Study design and setting: Our metaepidemiological study included all DTA reviews in the Cochrane Library up to October 6th, 2019. We summarized reviews' characteristics focusing on heterogeneity analysis. We selected reviews with a bivariate model and ≥4 studies for reanalysis. In this group, we fitted bivariate random effects models and we quantified heterogeneity by means of logit variances of sensitivity and specificity, bivariate I², median odds ratio (OR), and the area of the 95% prediction ellipse. We provided a narrative interpretation of these measures in different scenarios.

Results: There were 124 Cochrane DTA reviews of which 91 (73%) included meta-analysis. Only in 5 meta-analyses, variances of the logit sensitivity and specificity were reported, and in 21 meta-analyses (23%), the 95% prediction ellipse was reported without any calculation of its area. We selected 60 of these 91 reviews to explore the behavior of all measures of heterogeneity. We found that most reviews described the subjective heterogeneity as moderate or extreme (n = 31/60, 52%), whereas the area of the 95% prediction ellipse and the median OR for sensitivity and specificity showed high variability; the area ranged from 5% to 97%, the median OR of sensitivity ranged from 1.13 to 10.7, and the median OROR of specificity ranged from 1.18 to 19.68.

Conclusion: Cochrane DTA reviews show a poor reporting of between-study heterogeneity. Using median OR and the area of the 95% prediction ellipse will improve reporting and interpretation of this crucial aspect of DTA meta-analysis.

Keywords: Diagnostic test accuracy; Heterogeneity; Meta-analysis; Systematic review.