Transfer Learning Approaches for Neuroimaging Analysis: A Scoping Review

Abstract

Deep learning algorithms have been moderately successful in diagnoses of diseases by analyzing medical images especially through neuroimaging that is rich in annotated data. Transfer learning methods have demonstrated strong performance in tackling annotated data. It utilizes and transfers knowledge learned from a source domain to target domain even when the dataset is small. There are multiple approaches to transfer learning that result in a range of performance estimates in diagnosis, detection, and classification of clinical problems. Therefore, in this paper, we reviewed transfer learning approaches, their design attributes, and their applications to neuroimaging problems. We reviewed two main literature databases and included the most relevant studies using predefined inclusion criteria. Among 50 reviewed studies, more than half of them are on transfer learning for Alzheimer's disease. Brain mapping and brain tumor detection were second and third most discussed research problems, respectively. The most common source dataset for transfer learning was ImageNet, which is not a neuroimaging dataset. This suggests that the majority of studies preferred pre-trained models instead of training their own model on a neuroimaging dataset. Although, about one third of studies designed their own architecture, most studies used existing Convolutional Neural Network architectures. Magnetic Resonance Imaging was the most common imaging modality. In almost all studies, transfer learning contributed to better performance in diagnosis, classification, segmentation of different neuroimaging diseases and problems, than methods without transfer learning. Among different transfer learning approaches, fine-tuning all convolutional and fully-connected layers approach and freezing convolutional layers and fine-tuning fully-connected layers approach demonstrated superior performance in terms of accuracy. These recent transfer learning approaches not only show great performance but also require less computational resources and time.

Keywords: convolutional neural network; domain adaptation; fine tuning; medical imaging; neuroimaging; transfer learning.

Figure 1

The LeNet architecture for letter recognition, one of the first CNN architectures for image processing. FC, Fully-connected layer. The architecture was designed for handwritten digit recognition.

Figure 2

Demonstration of transferring weights of a convolution filter from source domain to target domain.

Figure 3

Flowchart illustrating literature search process and extraction of studies meeting the scoping review inclusion criteria.

Figure 4

Residual block of the ResNet algorithm (left) and the ResNet 12 architecture (right).

Figure 5

The GoogLeNet inception modules. Left: Naïve version of inception module. Right: Inception module with dimensionality reduction.

Figure 6

The AlexNet architecture.