Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 17:12:100562.
doi: 10.1016/j.ejro.2024.100562. eCollection 2024 Jun.

Reproducing RECIST lesion selection via machine learning: Insights into intra and inter-radiologist variation

Teresa M Tareco Bucho  1   2 , Liliana Petrychenko  1   2 , Mohamed A Abdelatty  1   3 , Nino Bogveradze  1   2   4 , Zuhir Bodalal  1   2 , Regina G H Beets-Tan  1   2   5 , Stefano Trebeschi  1   2
Affiliations

Reproducing RECIST lesion selection via machine learning: Insights into intra and inter-radiologist variation

Teresa M Tareco Bucho et al. Eur J Radiol Open. .

Abstract

Background: The Response Evaluation Criteria in Solid Tumors (RECIST) aims to provide a standardized approach to assess treatment response in solid tumors. However, discrepancies in the selection of measurable and target lesions among radiologists using these criteria pose a significant limitation to their reproducibility and accuracy. This study aimed to understand the factors contributing to this variability.

Methods: Machine learning models were used to replicate, in parallel, the selection process of measurable and target lesions by two radiologists in a cohort of 40 patients from an internal pan-cancer dataset. The models were trained on lesion characteristics such as size, shape, texture, rank, and proximity to other lesions. Ablation experiments were conducted to evaluate the impact of lesion diameter, volume, and rank on the selection process.

Results: The models successfully reproduced the selection of measurable lesions, relying primarily on size-related features. Similarly, the models reproduced target lesion selection, relying mostly on lesion rank. Beyond these features, the importance placed by different radiologists on different visual characteristics can vary, specifically when choosing target lesions. Worth noting that substantial variability was still observed between radiologists in both measurable and target lesion selection.

Conclusions: Despite the successful replication of lesion selection, our results still revealed significant inter-radiologist disagreement. This underscores the necessity for more precise guidelines to standardize lesion selection processes and minimize reliance on individual interpretation and experience as a means to bridge existing ambiguities.

Keywords: Cancer imaging; Machine learning; RECIST; Reproducibility; Variability.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Fig. 1
Fig. 1
Schematic representation of the data analysis. Icons from flaticon.com.
Fig. 2
Fig. 2
Performance of the models in measurable and target lesion selection of the models across the 100 MCCV test folds, on an overall and patient-level, for the across different ablation experiments and settings. Nn: not-significant; ***: p-value < 0.001.
Fig. 3
Fig. 3
Overview of the dataset with the measurable lesions labeling from the readers. Color encodes the agreement between readers: green in case of agreement, yellow for lesions selected by reader 1, blue for lesions selected by reader 2.
Fig. 4
Fig. 4
Comparative analysis of the models’ performances in the three ablation experiments between readers for target lesion selection.
Fig. 5
Fig. 5
Correlation of the feature importance between reader 1 and reader 2 models, for the selection of (A) measurable lesions, (B) target lesions, and (C) target lesions with rank ablation.
See this image and copyright information in PMC

References

    1. Beaumont Hubert, Evans Tracey L., Klifa Catherine, Guermazi Ali, Hong Sae Rom, Chadjaa Mustapha, Monostori Zsuzsanna. Discrepancies of assessments in a RECIST 1.1 Phase II Clinical Trial – association between adjudication rate and variability in images and tumors selection. Cancer Imaging. 2018;18(1):50. doi: 10.1186/s40644-018-0186-0. - DOI - PMC - PubMed
    1. Coy Heidi J., Douek Michael L., Ruchalski Kathleen, Kim Hyun J., Gutierrez Antonio, Patel Maitrya, Sai Victor, et al. Components of radiologic progressive disease defined by RECIST 1.1 in patients with metastatic clear cell renal cell carcinoma. Radiology. 2019;292(1):103–109. doi: 10.1148/radiol.2019182922. - DOI - PubMed
    1. Einstein Gilles O., McDaniel Mark A. Normal aging and prospective memory. J. Exp. Psychol. Learn. Mem., Cogn. 1990;16(4):717–726. doi: 10.1037/0278-7393.16.4.717. - DOI - PubMed
    1. Eisenhauer E.A., Therasse P., Bogaerts J., Schwartz L.H., Sargent D., Ford R., Dancey J., et al. New response evaluation criteria in solid tumours: revised RECIST guideline (Version 1.1) Eur. J. Cancer. 2009;45(2):228–247. doi: 10.1016/j.ejca.2008.10.026. - DOI - PubMed
    1. Goldmacher Gregory V., Conklin James. The use of tumour volumetrics to assess response to therapy in anticancer clinical trials: tumour volumetrics in clinical trials. Br. J. Clin. Pharmacol. 2012;73(6):846–854. doi: 10.1111/j.1365-2125.2012.04179.x. - DOI - PMC - PubMed

LinkOut - more resources