Evaluation of cancer outcome assessment using MRI: A review of deep-learning methods

Abstract

Accurate evaluation of tumor response to treatment is critical to allow personalized treatment regimens according to the predicted response and to support clinical trials investigating new therapeutic agents by providing them with an accurate response indicator. Recent advances in medical imaging, computer hardware, and machine-learning algorithms have resulted in the increased use of these tools in the field of medicine as a whole and specifically in cancer imaging for detection and characterization of malignant lesions, prognosis, and assessment of treatment response. Among the currently available imaging techniques, magnetic resonance imaging (MRI) plays an important role in the evaluation of treatment assessment of many cancers, given its superior soft-tissue contrast and its ability to allow multiplanar imaging and functional evaluation. In recent years, deep learning (DL) has become an active area of research, paving the way for computer-assisted clinical and radiological decision support. DL can uncover associations between imaging features that cannot be visually identified by the naked eye and pertinent clinical outcomes. The aim of this review is to highlight the use of DL in the evaluation of tumor response assessed on MRI. In this review, we will first provide an overview of common DL architectures used in medical imaging research in general. Then, we will review the studies to date that have applied DL to magnetic resonance imaging for the task of treatment response assessment. Finally, we will discuss the challenges and opportunities of using DL within the clinical workflow.

Figure 1.

A general framework (based on deep learning algorithms) for the processing of images by classifying high and low risk of survival, thus assessing probability of treatment response. (A) Patient MRI images are input into a deep learning model for the purpose of training the model; (B) A deep-learning system developed and trained to characterize outcome assessment, such as survival probability; (C) The outcome of the deep-learning model is used to predict cancer outcome.

Figure 2.

Illustration of an artificial neural networks (ANNs), the backbone of deep neural networks (DNNs). In this figure, we show a fully connected neural network where all the nodes, or neurons, in one layer are connected to the neurons in the next layer. When the input increases, fully connected networks tend to be computationally expensive, resulting in poor scalability.

Figure 3.

An illustration of a simple convolutional neural network including convolutional, pooling, and fully connected layers. The two-dimensional input data undergoe multiple rounds of convolution and subsample layers. Feature extraction by filters are learned through back projection. The pooling operations, including max or mean, in a region are used to reduce the number of pixels in each layer of the network. Each operation increasingly extracts higher order discriminative features. Ultimately, the output layer is a class probability based on these higher order features.

Figure 4.

The U-net network structure has a deep-learning encoder-decoder architecture. The CNN is termed “U-net” due to the u-shaped structure. The network consists of encoder layers where there is first downsampling in the image size followed by upsampling in the expansive or decoder layer.

Figure 5.

The VGG-16 architecture. The VGG16 consists of 13 convolutional layers, five max-pooling layers, and three fully connected layers. Consequently, the number of tunable parameters is 16 (13 convolutional layers and three fully connected layers).

Figure 6.

Illustration for the architecture of recurrent neural network (RNN). RNNs are a class of neural network commonly used for text and sequence data. They allow previous outputs to be used outputs to be used as inputs while having hidden states. An important class of RNNs are long-short-term memory (LSTM) which have feedback connections are often used for time series analysis.

Figure 7.

Illustration of a basic autoencoder. An autoencoder is an unsupervised learning model assigned the task of transforming the input image into a latent or compressed representation by minimizing the reconstruction errors between input and reconstructed images of the network. An autoencoder performs two tasks. It first encodes an image, and subsequently it decodes it. Encoding an image in this context means that the autoencoder generates a compressed representation of the original image. Conversely, the decoder takes the output from the bottle neck (latent space representation) and attempts to recreate the input image. For the autoencoder to reconstruct an image, it will need to learn some latent representation of the image. Latent representation refers to a set of compressed features of the image which are learned by the network through an iterative process of training, and which are subsequently used to reconstruct the desired image.

Figure 8.

In Generative adversarial networks (GANs) consists of two models: the discriminator and the generator. GANs learn through deriving backpropagation signals through a competitive process involving a pair of networks.