NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals

Abstract

Purpose: The purpose of this study was to conduct a meta-research of radiomics-related articles for the publication of negative results, with a focus on the leading clinical radiology journals due to their purportedly high editorial standards.

Methods: A literature search was performed in PubMed to identify original research studies on radiomics (last search date: August 16th, 2022). The search was restricted to studies published in Q1 clinical radiology journals indexed by Scopus and Web of Science. Following an a priori power analysis based on our null hypothesis, a random sampling of the published literature was conducted. Besides the six baseline study characteristics, a total of three items about publication bias were evaluated. Agreement between raters was analyzed. Disagreements were resolved through consensus. Statistical synthesis of the qualitative evaluations was presented.

Results: Following a priori power analysis, we included a random sample of 149 publications in this study. Most of the publications were retrospective (95%; 142/149), based on private data (91%; 136/149), centered on a single institution (75%; 111/149), and lacked external validation (81%; 121/149). Slightly fewer than half (44%; 66/149) made no comparison to non-radiomic approaches. Overall, only one study (1%; 1/149) reported negative results for radiomics, yielding a statistically significant binomial test (p < 0.0001).

Conclusion: The top clinical radiology journals almost never publish negative results, having a strong bias toward publishing positive results. Almost half of the publications did not even compare their approach with a non-radiomic method.

Keywords: Meta-research; Negative results; Publication bias; Radiomics; Systematic review; Texture analysis.