Swin Transformer and the Unet Architecture to Correct Motion Artifacts in Magnetic Resonance Image Reconstruction

Abstract

We present a deep learning-based method that corrects motion artifacts and thus accelerates data acquisition and reconstruction of magnetic resonance images. The novel model, the Motion Artifact Correction by Swin Network (MACS-Net), uses a Swin transformer layer as the fundamental block and the Unet architecture as the neural network backbone. We employ a hierarchical transformer with shifted windows to extract multiscale contextual features during encoding. A new dual upsampling technique is employed to enhance the spatial resolutions of feature maps in the Swin transformer-based decoder layer. A raw magnetic resonance imaging dataset is used for network training and testing; the data contain various motion artifacts with ground truth images of the same subjects. The results were compared to six state-of-the-art MRI image motion correction methods using two types of motions. When motions were brief (within 5 s), the method reduced the average normalized root mean square error (NRMSE) from 45.25% to 17.51%, increased the mean structural similarity index measure (SSIM) from 79.43% to 91.72%, and increased the peak signal-to-noise ratio (PSNR) from 18.24 to 26.57 dB. Similarly, when motions were extended from 5 to 10 s, our approach decreased the average NRMSE from 60.30% to 21.04%, improved the mean SSIM from 33.86% to 90.33%, and increased the PSNR from 15.64 to 24.99 dB. The anatomical structures of the corrected images and the motion-free brain data were similar.

Figure 1

The receptive fields for two-dimensional convolution (Conv2D), multihead self-attention (MSA), and shifted window-based MSA (W-MSA/SW-MSA) are shown in (a) and (b). Receptive field of operation, green box; pixel, yellow box; self-attention patch, red box.

Figure 2

The Swin transformer-based method for correction of MAs. C: number of channels; GELU: Gaussian error linear unit.

Figure 3

The new dual upsampling module employing both subpixel and bilinear approaches.

Figure 4

A Swin transformer block (STB) with eight Swin transformer layers (STLs).

Figure 5

Two Swin transformer layers (STLs).

Figure 6

Training and validation losses of the proposed method.

Figure 7

PSNR data acquired during model training.

Figure 8

Reconstructed images of slice 82 of the test dataset with brief motion (<5 s). (a) The motion-free ground truth image. (b) The image with an MA. Images corrected by (c) MoCo-Net, (d) Namer-Net, (e) Modified-2D-Net, (f) MC-Net, (g) Stacked-Unet, (h) Mark-Net, and (i) MACS-Net.

Figure 9

Reconstructed images of slice 82 of the test dataset with moderate brief motion (5–10 s). (a) Motion-free ground truth image. (b) The image with an MA. Images corrected by (c) MoCo-Net, (d) Namer-Net, (e) Modified-2D-Net, (f) MC-Net, (g) Stacked-Unet, (h) Mark-Net, and (i) MACS-Net.