Deep learning for accelerated and robust MRI reconstruction

Abstract

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

Keywords: Deep learning; Image reconstruction; MRI; Machine learning.

Fig. 1

Overview of the DL-based MRI reconstruction landscape. While many different methods are available, and those often incorporate elements from other techniques, we classify them into five main categories

Fig. 2

Elimination of hardware artifacts at low field MRI (6.5 mT) using AUTOMAP. Two slices from a 3D bSSFP (NA = 50) are shown. When reconstructed with IFFT (a, b), a vertical artifact (red arrows) is present across slices. When the same raw data was reconstructed with AUTOMAP (c, d), the artifacts are eliminated. The error maps of each slice with respect to a reference scan (NA = 100) is shown for both IFFT and AUTOMAP reconstruction. e–g Uncorrupted k-space (NA = 50) was reconstructed with AUTOMAP (e) and IFFT (f). Adapted and modified from Koonjoo, N. et al. Sci Rep 11, 8248 (2021). https://doi.org/10.1038/s41598-021-87482-7 [56]

Fig. 3

Variational network (VN) training procedure. The objective is learning a set of VN parameters during an offline training procedure. For this purpose, the current reconstruction of the VN is compared to an artifact-free reference using a similarity measure. This yields the reconstruction error which is propagated back to the VN to compute a new set of parameters. Reproduced with permission from Hammernik, K. et al. (2018) Magn. Reson. Med., 79: 3055–3071. https://doi.org/10.1002/mrm.26977. [24]

Fig. 4

Example of the input training data for three DL reconstruction methods. The fully-supervised MoDL method [25] receives var-dens sampled data as input and uses the entire k-space for supervision. The self-supervised SSDU method [132] receives var-dens data as input, splits it into two subsets, and uses one set for data consistency and the other for supervision. In this example, the var-dens data were sampled from parallel-imaging (equispaced) acquired data, as in [132]. The k-band method [135] receives var-dens sampled data from a k-space band, and uses data from the whole band for supervision, without any supervision outside the band. Different bands are acquired from different subjects, with random orientations. At inference, the input to all three methods is var-dens data from the entire k-space, similar to that shown here for MoDL

Fig. 5. Automated discovery of MRI acquisition protocols using supervised learning

. A differentiable MR scanner utilizes the Bloch equations for in-silico signal generation and the later reconstruction of the target contrast of interest from real, acquired data. Reproduced from Loktyushin et al. Magn. Reson. Med. 2021; 86: 709-724 [186]

Fig. 6. A demonstration of two MRI quantification strategies/architectures.

a Deep learning reconstruction of quantitative magnetic resonance fingerprinting (MRF) information. A fully connected neural network is trained using simulated signal trajectories. During inference, it receives a series of raw MRF images pixel-wise (gray-scale images, left), as well as auxiliary maps (color, top left), yielding quantitative parameter maps (top right). b A further acceleration in quantitative MRI scan time can be achieved by training a generative adversarial network (GAN) using a smaller subset of raw input data to yield the same quantitative output maps. Reproduced and modified from Weigand-Whittier et al. [199]

Fig. 7. Impact of common image perturbations on image quality metrics

. A variety of image perturbations applied to a sample image from the fastMRI dataset (top row: noise addition, image blurring, pixel rolling (where an image is shifted by a number of pixels), and physics-based subject motion. The impact of these corruptions is shown for conventional image quality metrics (SSIM, PSNR) and deep feature distance metrics (LPIPS—made for natural images, SSFD—made for MR images). The deep feature metrics exhibit a larger dynamic range to the noise, blurring, and motion corruptions, but present very little change due to pixel rolling, since the image quality does not change. These qualities of deep feature metrics are ideal for assessing MRI reconstruction quality