Convolutional Neural Network Based Frameworks for Fast Automatic Segmentation of Thalamic Nuclei from Native and Synthesized Contrast Structural MRI

Abstract

Thalamic nuclei have been implicated in several neurological diseases. Thalamic nuclei parcellation from structural MRI is challenging due to poor intra-thalamic nuclear contrast while methods based on diffusion and functional MRI are affected by limited spatial resolution and image distortion. Existing multi-atlas based techniques are often computationally intensive and time-consuming. In this work, we propose a 3D convolutional neural network (CNN) based framework for thalamic nuclei parcellation using T1-weighted Magnetization Prepared Rapid Gradient Echo (MPRAGE) images. Transformation of images to an efficient representation has been proposed to improve the performance of subsequent classification tasks especially when working with limited labeled data. We investigate this by transforming the MPRAGE images to White-Matter-nulled MPRAGE (WMn-MPRAGE) contrast, previously shown to exhibit good intra-thalamic nuclear contrast, prior to the segmentation step. We trained two 3D segmentation frameworks using MPRAGE images (n = 35 subjects): (a) a native contrast segmentation (NCS) on MPRAGE images and (b) a synthesized contrast segmentation (SCS) where synthesized WMn-MPRAGE representation generated by a contrast synthesis CNN were used. Thalamic nuclei labels were generated using THOMAS, a multi-atlas segmentation technique proposed for WMn-MPRAGE images. The segmentation accuracy and clinical utility were evaluated on a healthy cohort (n = 12) and a cohort (n = 45) comprising of healthy subjects and patients with alcohol use disorder (AUD), respectively. Both the segmentation CNNs yielded comparable performances on most thalamic nuclei with Dice scores greater than 0.84 for larger nuclei and at least 0.7 for smaller nuclei. However, for some nuclei, the SCS CNN yielded significant improvements in Dice scores (medial geniculate nucleus, P = 0.003, centromedian nucleus, P = 0.01) and percent volume difference (ventral anterior, P = 0.001, ventral posterior lateral, P = 0.01) over NCS. In the AUD cohort, the SCS CNN demonstrated a significant atrophy in ventral lateral posterior nucleus in AUD patients compared to healthy age-matched controls (P = 0.01), agreeing with previous studies on thalamic atrophy in alcoholism, whereas the NCS CNN showed spurious atrophy of the ventral posterior lateral nucleus. CNN-based segmentation of thalamic nuclei provides a fast and automated technique for thalamic nuclei prediction in MPRAGE images. The transformation of images to an efficient representation, such as WMn-MPRAGE, can provide further improvements in segmentation performance.

Keywords: 3D convolutional neural networks; Alcohol use disorder; Contrast synthesis; Representational learning; Thalamic-nuclei segmentation; White-matter-nulled MPRAGE.

Fig. 1:

Overview of the proposed multi-scale thalamic nuclei segmentation frameworks. (a) The thalamus and its individual nuclei are predicted directly on MPRAGE images using the Native Contrast Segmentation (NCS) framework. (b) A contrast synthesis CNN is introduced prior to the segmentation CNN for synthesized contrast segmentation (SCS) framework. This synthesis CNN learns a new representation (WMn-MPRAGE) from MPRAGE contrast. The ground truth labels are generated on the WMn-MPRAGE images using a multi-atlas label fusion algorithm (THOMAS).

Fig. 2:

Convolutional Neural Network (CNN) architectures for synthesis and segmentation. (a) A 3D multi-scale thalamic nuclei segmentation CNN and (b) a 3D multi-scale white-matter-nulled contrast synthesis CNN are shown. The number of feature maps generated from each layer are denoted next to the layer. Note that both architectures have a similar encoder-decoder like framework with skip connections except for the choice of loss functions.

Fig. 3:

White-matter-nulled MPRAGE (WMn-MPRAGE) synthesis. Representative axial and sagittal cross-sections of a synthesized image from a test subject are shown. The target WMn-MPRAGE images as well as input MPRAGE images are also shown for reference. The synthesized WMn-MPRAGE images are able to mimic the contrast between thalamus and the surrounding white matter similar to the WMn-MPRAGE images.

Fig. 4:

Thalamic nuclei prediction on MPRAGE and synthesized WMn-MPRAGE images using NCS and the SCS CNN frameworks, respectively. Top row (a-b): Representative axial cross-sections from a test subject for the different image contrast are shown. It is to be noted that synthesized WMn-MPRAGE contrast shows improved inter-thalamic contrast. The arrows in (b) highlight nuclei boundaries on synthesized WMn-MPRAGE image for MD and Pul nuclei. Bottom row: Individual thalamic nuclei labels predictions from the (c) MPRAGE contrast and (d) synthesized WMn-MPRAGE contrast are overlaid on the images with reference THOMAS labels outlined on the left thalamus. The segmentation CNN using synthesized contrast is able to accurately delineate even smaller nuclei such as ventral anterior (VA), habenula (Hb), and mammillothalamic tract (MTT).

Fig. 5:

Thalamic nuclei predictions from the synthesized contrast segmentation CNN. Multiplanar views of the test subject (top row) in Figure 4 with individual nuclei labels from the antral, ventral, posterior, and medial thalamic nuclei groups predicted using synthesized WMn-MPRAGE (bottom row) are shown. The corresponding ground truth labels from THOMAS are overlaid (color outlines) for the left thalamus. The coronal view shows the LGN and MGN nuclei. The predicted labels for the smaller nuclei such as AV, VA, VLa, CM, LGN, and MGN agree well with the THOMAS outlines.

Fig. 6:

Bland-Altman plots to assess agreement between predicted volumes and ground truth THOMAS labels on the Alcohol Use Disorder (AUD) cohort. Bland-Altman plots for agreement in volumes for Thalamus (a), ventral lateral posterior (b), and ventral posterior lateral nucleus (c) for the segmentation CNN trained on MPRAGE images (top row) are shown. The plots are also generated for the volumes predicted by the segmentation CNN trained on synthesized WMn-MPRAGE images (bottom row). The subjects are color-coded with blue for controls and red for subjects with AUD. For each case, Pearson’s correlation coefficient (ρ), percent repeatability coefficient (RPC), and coefficient of variation (CV) are also reported. It should be noted that on the AUD cohort, the volumes from synthesized WMn-MPRAGE images have tighter limits of agreement. Additionally, the NCS framework underestimates the volumes for thalamus and ventral posterior lateral nucleus for subjects with AUD (red).

Fig. 7:

Comparison of thalamic nuclei segmentation CNN performance with different image contrasts on the test cohorts (test cohort 1 and alcohol use disorder cohort). The bar plots compare the improvements in Dice score (a) and volume difference (VD) (b) for the different nuclei when using synthesized WMn-MPRAGE and WMn-MPRAGE over conventional MPRAGE contrast. The nuclei are arranged in decreasing order of their average volumes. Transforming the MPRAGE contrast to synthesized WMn-MPRAGE representation yields improvements in segmentation performance. The usage of true WMn-MPRAGE contrast outperforms MPRAGE based segmentation.

Fig. 8:

Thalamic nuclei prediction on a subject with lesions in the thalamus. Axial cross-sections of the subject, with alcohol use disorder, are shown in the top row (a-b). The thalamic lesions (yellow arrows) appear bright in the WM-nulled representation of MPRAGE image. The predictions from the corresponding NCS CNN (c) and SCS CNN (d) are shown. On this particular case, the segmentation using MPRAGE contrast (NCS) fails to correctly delineate nuclei surrounding the lesions, especially centromedian (CM), pulvinar (Pul), and ventral posterior lateral (VPl) nuclei.