A critical assessment of artificial intelligence in magnetic resonance imaging of cancer

Abstract

Given the enormous output and pace of development of artificial intelligence (AI) methods in medical imaging, it can be challenging to identify the true success stories to determine the state-of-the-art of the field. This report seeks to provide the magnetic resonance imaging (MRI) community with an initial guide into the major areas in which the methods of AI are contributing to MRI in oncology. After a general introduction to artificial intelligence, we proceed to discuss the successes and current limitations of AI in MRI when used for image acquisition, reconstruction, registration, and segmentation, as well as its utility for assisting in diagnostic and prognostic settings. Within each section, we attempt to present a balanced summary by first presenting common techniques, state of readiness, current clinical needs, and barriers to practical deployment in the clinical setting. We conclude by presenting areas in which new advances must be realized to address questions regarding generalizability, quality assurance and control, and uncertainty quantification when applying MRI to cancer to maintain patient safety and practical utility.

Keywords: Biomedical engineering; Cancer imaging; Image processing.

Fig. 1. Example reconstructions from the 2020 FastMRI Challenge, adapted from ref. .

The fully sampled images (A, C) were retrospectively subsampled to simulate 8× (top) and 4× (bottom) faster scans. In the top case, the DL reconstruction (B) is able to reproduce with high fidelity the lesion in the post-contrast T₁-weighted image, though with some blurring. In the bottom case, the DL reconstruction (D) hallucinated a false vessel (red arrow), perhaps due to the surgical staple artifact not being well-represented in the training set.

Fig. 2. The flow chart depicts the downstream impact that quantitative imaging workflow can have on clinical applications.

Beginning with image acquisition (A), variability and uncertainty is propagated through each step and can affect the accuracy in assessing lesion response and/or anticipated clinical outcome (H). Image processing routines (B) are then used to quantify image features prior to image registration (C), however, image registration may also occur prior to post-processing. Once the images are co-registered, automatic or manual segmentation (D) identifies the tissue of interest. At this stage, the clinical response can be assessed by determining lesion response (E). Alternatively, the features (F) from imaging and -omics (G) can be identified to predict clinical outcome (H) or lesion response (E). Error at any of these steps can compound and result in a false classification of patient outcomes.

Fig. 3. AI for imaging-based cancer detection and diagnosis.

The figure illustrates the global process whereby MRI sequences (i.e., T₂-weighted (T2W), DW-MRI, and DCE-MRI) are processed with a CNN to identify whether there is a tumor present, its localization, and the differential diagnosis (e.g., tumor class, tumor subtype, clinical risk). Several CNN architectures have been used to construct CADe and CADx systems for this type of MRI analysis^,,,,. This figure shows the use of a U-net due to its successful use for automatic tumor detection and segmentation, as well as in the context of cancer diagnosis.

Fig. 4. Demonstration of how an SVM and a CNN can use imaging data to predict treatment response.

In Panel A, features related to histograms of relative cerebral blood volume or peak height of a perfusion signal are extracted from the imaging data. Clinical information such as patient age and genetic data can also be included as features. The goal of the SVM is to take N features and determine the (N-1)-dimensional hyperplanes that maximally separate (for example) patients into short, medium, or long survival, or complete response as determined by pathology. Panel B shows potential inputs to a CNN – either the whole image domain, imaging-derived features, or a patch of the domain. Extracting patches from a domain can be used to increase the amount of training data, or to reduce computational burden when working with large images. These are then input to a CNN, here represented with convolution and down sampling layers feeding into a fully connected architecture. In general, multiple sets of convolution and down-sampling layers are used. The network output accomplishes the same goal as the SVM in Panel A; namely, separating inputs into classes such as responders and non-responders, or survival at a particular time.