Transfer learning in deep neural network based under-sampled MR image reconstruction

Abstract

In Magnetic Resonance Imaging (MRI), the success of deep learning-based under-sampled MR image reconstruction depends on: (i) size of the training dataset, (ii) generalization capabilities of the trained neural network. Whenever there is a mismatch between the training and testing data, there is a need to retrain the neural network from scratch with thousands of MR images obtained using the same protocol. This may not be possible in MRI as it is costly and time consuming to acquire data. In this research, a transfer learning approach i.e. end-to-end fine tuning is proposed for U-Net to address the data scarcity and generalization problems of deep learning-based MR image reconstruction. First the generalization capabilities of a pre-trained U-Net (initially trained on the human brain images of 1.5 T scanner) are assessed for: (a) MR images acquired from MRI scanners of different magnetic field strengths, (b) MR images of different anatomies and (c) MR images under-sampled by different acceleration factors. Later, end-to-end fine tuning of the pre-trained U-Net is proposed for the reconstruction of the above-mentioned MR images (i.e. (a), (b) and (c)). The results show successful reconstructions obtained from the proposed method as reflected by the Structural SIMilarity index, Root Mean Square Error, Peak Signal-to-Noise Ratio and central line profile of the reconstructed images.

Keywords: CG-SENSE; Data scarcity; Deep learning; Generalization capabilities; MRI; Transfer learning.