Role of deep learning in infant brain MRI analysis

Abstract

Deep learning algorithms and in particular convolutional networks have shown tremendous success in medical image analysis applications, though relatively few methods have been applied to infant MRI data due numerous inherent challenges such as inhomogenous tissue appearance across the image, considerable image intensity variability across the first year of life, and a low signal to noise setting. This paper presents methods addressing these challenges in two selected applications, specifically infant brain tissue segmentation at the isointense stage and presymptomatic disease prediction in neurodevelopmental disorders. Corresponding methods are reviewed and compared, and open issues are identified, namely low data size restrictions, class imbalance problems, and lack of interpretation of the resulting deep learning solutions. We discuss how existing solutions can be adapted to approach these issues as well as how generative models seem to be a particularly strong contender to address them.

Keywords: Convolutional neural networks; Deep learning; Infant MRI; Isointense segmentation; MRI; Machine learning; Prediction.

Figure 1:

T1w images of a typically-developing infant, scanned longitudinally at 0, 3, 6, 12 and 24 months of age.

Figure 2:

A typical pipeline for a machine learning approach for early prediction of neurodevelopmental disorders (NDDs) using infant structural MR brain images(sMRI). Please note that this workflow employs infant-specific processing and analysis steps.

Figure 3:

Venn diagram illustrating the relationship of deep learning, machine learning and artificial intelligence.

Figure 4:

Architectures of two feed-forward fully-connected neural networks. A classical neural network containing one hidden layer and a deep neural network (deep learning) that has two or more hidden layers. Modern deep neural networks can be based on tens to hundreds of hidden layers.

Figure 5:

Illustration of the two-stage 3D FCN architecture proposed by Zeng et al. [61] for the isointense infant brain segmentation in multi-modality MR images. The first stage FCN will produce an initial segmentation that in turn is used to model the spatial context map for each brain tissue using distance maps. The final segmentation is obtained in the second stage using both the spatial contact information and the original T1w and T2w images. Image courtesy of [61].

Figure 6:

An illustration of the ensemble of semi-dense 3D FCN proposed by Dolz et al. [63], where information from T1w and T2w images are fused using (a) early fusion strategy and (b) late fusion strategy. Image courtesy of [63].

Figure 7:

An example deep learning-based framework for the early prediction of NDDs using high-dimensional cortical measurements extracted from infant MRI brain images.

Figure 8:

The two-stage prediction pipeline proposed by Hazlett et al. [11] for early prediction of ASD.

Figure 9:

CNN extension to cortical surfaces proposed by Mostapha et. al [92]. Several surface convolutional blocks are applied to each hemisphere in parallel followed by fully connected and dropout layers.

Figure 10:

An alternative to the CNN extension presented in Mostapha et. al [92], surfaces can be re-meshed onto a completely regular structure called a geometry image [93] which allow learning using conventional CNN architectures.

Figure 11:

The surface-CNN architecture proposed by Mostapha et. al [92] was applied for the early prediction of ASD using features extracted from 6-month subcortical brain surfaces.

Figure 12:

The two most commonly used ways to perform multi-task learning (MTL) in deep neural networks.

Figure 13:

Illustration of some examples of the generative adversarial networks (GAN) variants utilized for different tasks in medical image analysis.