Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Apr;35(4):1980-1992.
doi: 10.1007/s00330-024-11057-z. Epub 2024 Sep 6.

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

Burak Kocak  1 , Ali Keles  2 , Fadime Kose  2 , Abdurrezzak Sendur  2
Affiliations

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

Burak Kocak et al. Eur Radiol. 2025 Apr.

Abstract

Purpose: This study aims to comprehensively evaluate the quality of radiomics research by examining unique papers from reviews using the radiomics quality score (RQS).

Methods: A literature search was conducted in PubMed (last search date: April 14, 2024). Systematic or non-systematic reviews using the RQS to evaluate radiomic studies were potentially included. Exclusion was applied at two levels: first, at the review level, and second, at the study level (i.e., for the individual articles previously evaluated within the reviews). Score-wise and item-wise analyses were performed, along with trend, multivariable, and subgroup analyses based on baseline study characteristics and validation methods.

Results: A total of 1574 unique papers (published online between 1999 and 2023) from 89 reviews were included in the final analysis. The median RQS percentage was 31% with an IQR of 25% (25th-75th percentiles, 14-39%). A positive correlation between median RQS percentage and publication year (2014-2023) was found, with Kendall's tau coefficient of 0.908 (p < 0.001), suggesting an improvement in quality over time. The quality of radiomics publications significantly varied according to different subfields of radiology (p < 0.001). Around one-third of the publications (32%) lacked a separate validation set. Papers with internal validation (54%) dominated those with external validation (14%). Higher-quality validation practices were significantly associated with better RQS percentage scores, independent of the validation's effect on the final score. Item-wise analysis revealed significant shortcomings in several areas.

Conclusion: Radiomics research quality is low but improving according to RQS. Significant variation exists across radiology subfields. Critical areas were identified for targeted improvement.

Clinical relevance statement: Our study shows that the quality of radiomics research is generally low but improving over time, with item-wise analysis highlighting critical areas needing improvement. It also reveals that the quality of radiomics research differs across subfields and validation methods.

Key points: Overall quality of radiomics research remains low and highly variable, although a significant positive trend suggests an improvement in quality over time. Considerable variations exist in the quality of radiomics publications across different subfields of radiology and validation types. The item-wise analysis highlights several critical areas requiring attention, emphasizing the need for targeted improvements.

Keywords: Artificial intelligence; Machine learning; Radiomics; Radiomics quality score; Systematic review.

PubMed Disclaimer

Conflict of interest statement

Compliance with ethical standards. Guarantor: The scientific guarantor of this publication is Burak Kocak, MD. Conflict of interest: B.K. is on the editorial board of European Radiology (section editor: Imaging Informatics and Artificial Intelligence). He has taken no part in this article’s peer review or selection. B.K. took part in the development of CLEAR and METRICS guidelines. The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article. Statistics and biometry: No complex statistical methods were necessary for this paper. Informed consent: Non-applicable. Ethical approval: Non-applicable. Study subjects or cohorts overlap: Non-applicable. Methodology: Systematic review

Comment in

References

    1. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446. https://doi.org/10.1016/j.ejca.2011.11.036 - DOI - PubMed - PMC
    1. Zhong J, Lu J, Zhang G et al (2023) An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation. Insights Imaging 14:111. https://doi.org/10.1186/s13244-023-01437-2 - DOI - PubMed - PMC
    1. Huang EP, O’Connor JPB, McShane LM et al (2023) Criteria for the translation of radiomics into clinically useful tests. Nat Rev Clin Oncol 20:69–82. https://doi.org/10.1038/s41571-022-00707-0 - DOI - PubMed
    1. Cobo M, Menéndez Fernández-Miranda P, Bastarrika G, Lloret Iglesias L (2023) Enhancing radiomics and deep learning systems through the standardization of medical imaging workflows. Sci Data 10:732. https://doi.org/10.1038/s41597-023-02641-x - DOI - PubMed - PMC
    1. Fusco R, Granata V, Grazzini G et al (2022) Radiomics in medical imaging: pitfalls and challenges in clinical management. Jpn J Radiol 40:919–929. https://doi.org/10.1007/s11604-022-01271-4 - DOI - PubMed

LinkOut - more resources