Overinterpretation of Research Findings: Evaluation of "Spin" in Systematic Reviews of Diagnostic Accuracy Studies in High-Impact Factor Journals

Abstract

Background: To compare the frequency of "spin" in systematic reviews of diagnostic accuracy studies in high-impact journals with the frequency a previously assessed series of reviews.

Methods: Medline was searched from January 2010 to January 2019. Systematic reviews of diagnostic accuracy studies were included if they reported a meta-analysis and were published in a journal with an impact factor >5. Two investigators independently scored each included systematic review for positivity of conclusions and for actual and potential overinterpretation practices.

Results: Of 137 included systematic reviews, actual overinterpretation was present in ≥1 form in the abstract in 63 (46%) and in the full-text report in 52 (38%); 108 (79%) contained a form of potential overinterpretation. Compared with the previously assessed series (reviews published 2015-2016), reviews in this series were less likely to contain ≥1 form of actual overinterpretation in the abstract and full-text report or ≥1 form of potential overinterpretation (P < 0.001 for all comparisons). The significance of these comparisons did not persist for actual overinterpretation in sensitivity analysis in which Cochrane systematic reviews were removed. Reviews published in the Cochrane Database of Systematic Reviews were less likely to contain actual overinterpretation in the abstract or the full-text report than reviews in other high-impact journals (P < 0.001 for both comparisons).

Conclusions: Reviews of diagnostic accuracy studies in high-impact journals are less likely to contain overinterpretation or spin. This difference is largely due to the reviews published in the Cochrane Database of Systematic Reviews, which contain spin less often than reviews published in other high-impact journals.