Artificial intelligence and machine learning in cancer imaging

Abstract

An increasing array of tools is being developed using artificial intelligence (AI) and machine learning (ML) for cancer imaging. The development of an optimal tool requires multidisciplinary engagement to ensure that the appropriate use case is met, as well as to undertake robust development and testing prior to its adoption into healthcare systems. This multidisciplinary review highlights key developments in the field. We discuss the challenges and opportunities of AI and ML in cancer imaging; considerations for the development of algorithms into tools that can be widely used and disseminated; and the development of the ecosystem needed to promote growth of AI and ML in cancer imaging.

Keywords: Biomarkers; Cancer imaging.

Fig. 1. Feature selection for radiomics.

In this illustration, a model classifier is shown to differentiate benign from malignant breast lesions on imaging. Initially, a large number of radiomic features were computed and after removing the highly correlated features, the zero and near-zero variance features; a recursive feature elimination and reduction method was applied. The model performance illustrated here identifies11 features to be at the saturation point. The red curve (left) is showing accuracy versus number of features, while the blue curve (right) represents the model’s error function over the number of features. In this example, using 11 imaging features shows high accuracy while minimising the error function.

Fig. 2. Quantitative Imaging Feature Pipeline.

This shows an example of the quantitative imaging feature pipeline (QIFP) used to process a positron emission tomography (PET) imaging cohort stored on a local network ePAD server. The box next to the “modify workflow” button is a selection button, which has been set to choose the workflow displayed. This workflow moves the image data into Stanford’s Quantitative Image Feature Engine (QIFE), which computes thousands of image features for each segmented tumour in the cohort, followed by a sparse regression modeler (LASSO TRAIN) that derives an association between a linear combination of a small number of image features to 5-year survival, and finally tests that model in an unseen cohort and produces an ROC curve displaying the accuracy of the association. Other workflows can be chosen that use one or more of the existing tools stored on the QIFP system.

Fig. 3. Potential use cases for artificial intelligence (AI) and machine learning (ML) in cancer imaging in relation to a patient’s cancer journey.

A typical asymptomatic patient eventually develops cancer presenting symptoms, which usually leads to the cancer diagnosis. Following appropriate disease staging, cancer treatment commences, which can lead to good response or even cure. However, some patients will relapse or progress on treatment for which additional treatment may be administered. Unfortunately, some patients will succumb to their disease. The potential uses for Imaging AI and ML are as shown at various stages of the cancer journey and discussed in the text.

Fig. 4. Machine Learning (ML) in a radiomics pipeline for evaluating tumour habitats.

a Whole tumour segmentation and identification of physiologically different regions by means of tissue-specific sub-segmentation on computed tomograhy (CT) imaging (e.g. using 3D volume rendering of tissue components with colour codes shown below). This is followed by b voxel-based radiomic feature map extraction and unsupervised clustering for tumour habitats considering the most clinically relevant region. Next, c quantitative measurements and inferred tumoural heterogeneity metrics are processed by ML predictive models to yield diagnostic and prognostic results. In this example, we have used CT images from a patient with metastatic ovarian cancer with a representative omental lesion.

Fig. 5. Potential future real-time tracking of whole tumour volume, spatial and temporal phenotypic heterogeneity with multi-omics data integration for precision oncology.

This schema would allow the processing of multi-institutional data, where each medical centre acquires and stores (in local PACS) its own medical imaging data. To execute quantitative analyses, a radiomics gateway is used to communicate outside the institution by requesting an automated, real-time tumour segmentation from a trusted and specialised AI/ML centre, which allows for continuous learning. The medical images leaving the hospital are anonymised to deal with cyber-security and privacy issues. The segmentation results are used for radiomic feature extraction and analysis, acting as virtual biopsies. The quantitative imaging results are integrated with other biomedical data streams to determine associations with clinical and multi-omics information. Such an approach may develop reliable diagnostic and prognostic tools for multidisciplinary team meetings to improve cancer care in clinical practice; and the evolution of precision oncology. PACS Picture Archiving and Communication System, ML Machine Learning.

Fig. 6. The Cancer Imaging Archive (TCIA) is a system of systems constructed from open-source software.

TCIA is also a set of services designed to collect and curate high quality cancer image data and related clinical data and make it publicly available. (VMs = virtual machines).