Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues

Abstract

Following the success of deep learning in a wide range of applications, neural network-based machine learning techniques have received interest as a means of accelerating magnetic resonance imaging (MRI). A number of ideas inspired by deep learning techniques from computer vision and image processing have been successfully applied to non-linear image reconstruction in the spirit of compressed sensing for both low dose computed tomography and accelerated MRI. The additional integration of multi-coil information to recover missing k-space lines in the MRI reconstruction process, is still studied less frequently, even though it is the de-facto standard for currently used accelerated MR acquisitions. This manuscript provides an overview of the recent machine learning approaches that have been proposed specifically for improving parallel imaging. A general background introduction to parallel MRI is given that is structured around the classical view of image space and k-space based methods. Both linear and non-linear methods are covered, followed by a discussion of recent efforts to further improve parallel imaging using machine learning, and specifically using artificial neural networks. Image-domain based techniques that introduce improved regularizers are covered as well as k-space based methods, where the focus is on better interpolation strategies using neural networks. Issues and open problems are discussed as well as recent efforts for producing open datasets and benchmarks for the community.

Keywords: Accelerated MRI; Deep learning; Iterative Image Reconstruction; Machine Learning; Numerical Optimization; Parallel Imaging.

Fig. 1:

In k-space based parallel imaging methods, missing data is recovered first in k-space, followed by an inverse discrete Fourier transform (IDFT) and combination of the individual coil elements. In image space based parallel imaging, the IDFT is performed as the first step, followed by coil sensitivity based removal of the aliasing artifacts from the reconstructed image by solving an inverse problem.

Fig. 2:

Illustration of machine learning-based image reconstruction. The network architecture consists of S stages that perform the equivalent of gradient descent steps in a classic iterative algorithm. Each stage consists of a regularizer and a data consistency layer. Training the network parameters Θ is performed by retrospectively undersampling fully sampled multi-coil raw k-space data and comparing the output of the network $u_d^S(\Theta )$ to a target reference reconstruction $u_d^{\text{ref }}$ obtained from the fully sampled data for the current training example d.

Fig. 3:

Comparison of image-domain based parallel imaging reconstructions of a retrospectively accelerated coronal knee acquisition. The used sampling pattern, zero-filling, CG-SENSE, combined parallel imaging and compressed sensing with a TGV-constraint, and learned reconstructions using the Variational Network and MoDL architectures are shown, along with their NRMSE and SSIM values to the fully sampled reference. See text in the respective sections for details on the individual experiments.

Fig. 4:

A slice from a high-resolution (0.6 mm isotropic) 7T brain acquisition, where all acquisitions were performed with prospective acceleration. It is difficult to acquire fully-sampled reference datasets for training for such acquisitions, thus two scan-specific k-space methods were compared. The CNN-based RAKI method visibly reduced noise amplification compared to the linear GRAPPA reconstruction. NL-GRAPPA and RAKI have similar noise properties, while RAKI produces a slightly sharper image at R = 6.

Fig. 5:

Comparison of k-space parallel imaging reconstructions of a retrospectively accelerated coronal knee acquisition, as in Figure 3. Due to the small size of the ACS data relative to the acceleration rate, the methods, none of which utilizes training databases, exhibit artifacts. GRAPPA has residual aliasing, whereas SPIRiT shows noise amplification. These are reduced in RAKI, though the residual artifacts remain. Respective NRMSE and SSIM values reflect these visual assessment.

Fig. 6:

Reconstruction results of simultaneous multi-slice imaging of 16 slices in fMRI (i.e. 16-fold acceleration in coverage), where a sample of 3 slices are shown. GRAPPA method exhibits noise amplification at this high acceleration rate. NL-GRAPPA reduces noise amplification but suffers from residual aliasing and leakage. The rRAKI method, which consists of a linear convolutional component G, in parallel with a non-linear CNN component F that learns the artifacts arising from G, exhibits exhibits reduced noise and reduced aliasing. Due to imperfections in the ACS data for this application, the residual component includes both noise amplification and residual artifacts.