Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice

Spyridon Bakas¹, Philipp Vollmuth², Norbert Galldiks³, Thomas C Booth⁴, Hugo J W L Aerts⁵, Wenya Linda Bi⁶, Benedikt Wiestler⁷, Pallavi Tiwari⁸, Sarthak Pati⁹, Ujjwal Baid¹⁰, Evan Calabrese¹¹, Philipp Lohmann¹², Martha Nowosielski¹³, Rajan Jain¹⁴, Rivka Colen¹⁵, Marwa Ismail⁸, Ghulam Rasool¹⁶, Janine M Lupo¹⁷, Hamed Akbari¹⁸, Joerg C Tonn¹⁹, David Macdonald²⁰, Michael Vogelbaum²¹, Susan M Chang²², Christos Davatzikos²³, Javier E Villanueva-Meyer¹⁷, Raymond Y Huang²⁴; Response Assessment in Neuro Oncology (RANO) group

Affiliations

¹ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA; Department of Radiology & Imaging Sciences, School of Medicine, Indiana University, Indianapolis, IN, USA; Department of Neurological Surgery, School of Medicine, Indiana University, Indianapolis, IN, USA; Department of Biostatistics and Health Data Science, School of Medicine, Indiana University, Indianapolis, IN, USA; Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianopolis, IN, USA; Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, IN, USA. Electronic address: spbakas@iu.edu.
² Division for Computational Radiology and Clinical AI, Clinic for Neuroradiology, University Hospital Bonn, Bonn, Germany; Faculty of Medicine, University of Bonn, Bonn, Germany; Division for Medical Image Computing, German Cancer Research Center, Heidelberg, Germany.
³ Department of Neurology, Faculty of Medicine and University Hospital Cologne, Cologne, Germany; Institute of Neuroscience and Medicine, Research Center Juelich, Juelich, Germany.
⁴ School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK; Department of Neuroradiology, King's College Hospital NHS Foundation Trust, London, UK.
⁵ Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, MA, USA; Radiology and Nuclear Medicine, Maastricht University, Maastricht, Netherlands.
⁶ Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
⁷ Department of Neuroradiology, University Hospital, Technical University of Munich, Munich, Germany.
⁸ Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA.
⁹ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA.
¹⁰ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA; Department of Radiology & Imaging Sciences, School of Medicine, Indiana University, Indianapolis, IN, USA; Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianopolis, IN, USA.
¹¹ Department of Radiology, School of Medicine, Duke University, Durham, NC, USA.
¹² Department of Neurology, Faculty of Medicine and University Hospital Cologne, Cologne, Germany; Department of Nuclear Medicine, University Hospital RWTH Aachen, Aachen, Germany.
¹³ Department of Neurology, Medical University Innsbruck, Innsbruck, Austria.
¹⁴ Department of Radiology and Department of Neurosurgery, NYU Grossman School of Medicine, New York, NY, USA.
¹⁵ Department of Radiology, Neuroradiology Division, Center for Artificial Intelligence Innovation in Medical Imaging, University of Pittsburgh, Pittsburgh, PA, USA.
¹⁶ Department of Machine Learning, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA.
¹⁷ Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA.
¹⁸ Department of Bioengineering, School of Engineering, Santa Clara University, Santa Clara, CA, USA.
¹⁹ Department of Neurosurgery, Ludwig-Maximilians-University, Munich, Germany; German Cancer Consortium, Partner Site Munich, Munich, Germany.
²⁰ London Regional Cancer Programme, London, ON, Canada.
²¹ Department of Neuro-Oncology, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA; Department of Neurosurgery, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA; H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA.
²² Department of Neurological Surgery, Division of Neuro-Oncology, University of California San Francisco, San Francisco, CA, USA.
²³ Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Center for Artificial Intelligence for Integrated Diagnostics and Center for Biomedical Image Computing and Analytics, University of Pennsylvania, Philadelphia, PA, USA.
²⁴ Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.

PMID: 39481415
PMCID: PMC12007431
DOI: 10.1016/S1470-2045(24)00315-2

Review

Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice

Spyridon Bakas et al. Lancet Oncol. 2024 Nov.

. 2024 Nov;25(11):e589-e601.

doi: 10.1016/S1470-2045(24)00315-2.

Authors

Affiliations

¹ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA; Department of Radiology & Imaging Sciences, School of Medicine, Indiana University, Indianapolis, IN, USA; Department of Neurological Surgery, School of Medicine, Indiana University, Indianapolis, IN, USA; Department of Biostatistics and Health Data Science, School of Medicine, Indiana University, Indianapolis, IN, USA; Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianopolis, IN, USA; Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, IN, USA. Electronic address: spbakas@iu.edu.
² Division for Computational Radiology and Clinical AI, Clinic for Neuroradiology, University Hospital Bonn, Bonn, Germany; Faculty of Medicine, University of Bonn, Bonn, Germany; Division for Medical Image Computing, German Cancer Research Center, Heidelberg, Germany.
³ Department of Neurology, Faculty of Medicine and University Hospital Cologne, Cologne, Germany; Institute of Neuroscience and Medicine, Research Center Juelich, Juelich, Germany.
⁴ School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK; Department of Neuroradiology, King's College Hospital NHS Foundation Trust, London, UK.
⁵ Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, MA, USA; Radiology and Nuclear Medicine, Maastricht University, Maastricht, Netherlands.
⁶ Department of Neurosurgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
⁷ Department of Neuroradiology, University Hospital, Technical University of Munich, Munich, Germany.
⁸ Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA.
⁹ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA.
¹⁰ Department of Pathology & Laboratory Medicine, Division of Computational Pathology, Indiana University, Indianopolis, IN, USA; Department of Radiology & Imaging Sciences, School of Medicine, Indiana University, Indianapolis, IN, USA; Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianopolis, IN, USA.
¹¹ Department of Radiology, School of Medicine, Duke University, Durham, NC, USA.
¹² Department of Neurology, Faculty of Medicine and University Hospital Cologne, Cologne, Germany; Department of Nuclear Medicine, University Hospital RWTH Aachen, Aachen, Germany.
¹³ Department of Neurology, Medical University Innsbruck, Innsbruck, Austria.
¹⁴ Department of Radiology and Department of Neurosurgery, NYU Grossman School of Medicine, New York, NY, USA.
¹⁵ Department of Radiology, Neuroradiology Division, Center for Artificial Intelligence Innovation in Medical Imaging, University of Pittsburgh, Pittsburgh, PA, USA.
¹⁶ Department of Machine Learning, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA.
¹⁷ Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA.
¹⁸ Department of Bioengineering, School of Engineering, Santa Clara University, Santa Clara, CA, USA.
¹⁹ Department of Neurosurgery, Ludwig-Maximilians-University, Munich, Germany; German Cancer Consortium, Partner Site Munich, Munich, Germany.
²⁰ London Regional Cancer Programme, London, ON, Canada.
²¹ Department of Neuro-Oncology, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA; Department of Neurosurgery, H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA; H Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA.
²² Department of Neurological Surgery, Division of Neuro-Oncology, University of California San Francisco, San Francisco, CA, USA.
²³ Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Center for Artificial Intelligence for Integrated Diagnostics and Center for Biomedical Image Computing and Analytics, University of Pennsylvania, Philadelphia, PA, USA.
²⁴ Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.

PMID: 39481415
PMCID: PMC12007431
DOI: 10.1016/S1470-2045(24)00315-2

Abstract

Technological advancements have enabled the extended investigation, development, and application of computational approaches in various domains, including health care. A burgeoning number of diagnostic, predictive, prognostic, and monitoring biomarkers are continuously being explored to improve clinical decision making in neuro-oncology. These advancements describe the increasing incorporation of artificial intelligence (AI) algorithms, including the use of radiomics. However, the broad applicability and clinical translation of AI are restricted by concerns about generalisability, reproducibility, scalability, and validation. This Policy Review intends to serve as the leading resource of recommendations for the standardisation and good clinical practice of AI approaches in health care, particularly in neuro-oncology. To this end, we investigate the repeatability, reproducibility, and stability of AI in response assessment in neuro-oncology in studies on factors affecting such computational approaches, and in publicly available open-source data and computational software tools facilitating these goals. The pathway for standardisation and validation of these approaches is discussed with the view of trustworthy AI enabling the next generation of clinical trials. We conclude with an outlook on the future of AI-enabled neuro-oncology.

PubMed Disclaimer

Conflict of interest statement

Declaration of interests NG reported Blue Earth Diagnostics honorarium and Telix Pharmaceuticals participation on Advisory Board. TCB reported consulting fees from Microvention and fees for speakers bureaus of Siemens Healthineers, Bayer, and Medtronic. HJWLA reported receiving grants from the National Institutes of Health, European Union, and the US Department of Veterans Affairs, and consulting fees from Onc.AI, Sphera, and Love Health. WLB reported receiving an honorarium from Stryker. PL reported receiving a grant from the German Research Foundation. RJ reported receiving a grant from AIRS Medical. JML reported receiving grants from the National Institutes of Health, Department of Defense, and GE Healthcare. JCT reported receiving grants from Novocure and Munich Surgical Imaging; royalties from Springer Publishing; consulting fees from Novartis; an honorarium from the Italian Society for Neurology; and support to attend meetings of the Belgian Association for Neurooncology and Glioma Meeting Athens 2023. MV reported receiving grants from Infuseon Therapeutics, Oncosynergy, and DeNovo Pharma; drug delivery device royalties; consulting fees from Servier Pharma and BioDexa Pharma; and holds patents for drug delivery devices. RYH reported consulting fees from Nuvation Bio and Vysioneet. All other authors declare no competing interests.

Figures

See this image and copyright information in PMC

References

1. Cagney DN, Sul J, Huang RY, Ligon KL, Wen PY, Alexander BM. The FDA NIH Biomarkers, EndpointS, and other Tools (BEST) resource in neuro-oncology. Neuro Oncol 2018; 20: 1162–72. - PMC - PubMed
1. Brennan CW, Verhaak RG, McKenna A, et al. The somatic genomic landscape of glioblastoma. Cell 2013; 155: 462–77. - PMC - PubMed
1. Sottoriva A, Spiteri I, Piccirillo SG, et al. Intratumor heterogeneity in human glioblastoma reflects cancer evolutionary dynamics. Proc Natl Acad Sci USA 2013; 110: 4009–14. - PMC - PubMed
1. Louis DN, Perry A, Wesseling P, et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro Oncol 2021; 23: 1231–51. - PMC - PubMed
1. Lambin P, Rios-Velazquez E, Leijenaar R, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012; 48: 441–46. - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice

Affiliations

Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources