Automatic cerebellum anatomical parcellation using U-Net with locally constrained optimization

Abstract

The cerebellum plays a central role in sensory input, voluntary motor action, and many neuropsychological functions and is involved in many brain diseases and neurological disorders. Cerebellar parcellation from magnetic resonance images provides a way to study regional cerebellar atrophy and also provides an anatomical map for functional imaging. In a recent comparison, a multi-atlas approach proved to be superior to other parcellation methods including some based on convolutional neural networks (CNNs) which have a considerable speed advantage. In this work, we developed an alternative CNN design for cerebellar parcellation, yielding a method that achieves the leading performance to date. The proposed method was evaluated on multiple data sets to show its broad applicability, and a Singularity container has been made publicly available.

Keywords: Cerebellum; Convolutional neural networks; Parcellation.

Figure 1:. Flowchart of ACAPULCO.

The cerebellum is parcellated by two CNNs: the locating network finds a bounding box around the cerebellum and the parcellating network labels the regions within the bounding box.

Figure 2:. Architectures of (A) the locating network, (B) the input block, and (C) the contracting block.

In (A), the number of output feature maps is marked within each block and the output spatial size is marked on the side.

Figure 3:. Architectures of (A) the parcellating network and (B) the expanding block.

The number of output feature maps is marked within each block and the output spatial size is marked on the side of contracting blocks. The output spatial sizes of the expanding blocks are the same as their corresponding contracting blocks.

Figure 4:. Comparison between before and after the post-processing:

(A) a coronal slice with the parcellating network output on the left and the the post-processing output on the right, and (B) another coronal slice of the same subject. Note that the post-processing can correct isolated mislabeling as indicated by the yellow arrows.

Figure 5:. Examples of data augmentation:

(A) the original, (B) the flipped, (C) the translated, (D) the scaled, (E) the rotated, and (F) the deformed images. The transformed label maps are plotted on top of the images.

Figure 6:. Dice scores of CERES2, CGCUTS, and ACAPULCO for the Adult Cohort.

Vertical axes are Dice scores. Dots represent testing images and bars represent their means. The mean Dice score across all regions for each testing image is shown in the last subfigure. ACAPULCO is not significantly different from CERES2, but scores best in terms of the mean Dice score (the bars in subfigures) in 18 out of 28 regions.

Figure 7:. Dice scores of CERES2, CGCUTS, and ACAPULCO for the Pediatric Cohort.

Vertical axes are Dice scores. Dots represent testing images and bars represent their means. The mean Dice score across all regions for each testing image is shown in the last subfigure. Five significantly different regions and the mean across all regions between CERES2 and ACAPULCO are marked by asterisks (p < 0.05 : * and p < 0.01 : **). ACAPULCO also scores best in terms of the mean Dice socre (the bars in subfigures) in 16 out of 18 regions.

Figure 8:. Coronal slices of three testing images of the Adult Cohort.

The left column is the images and the right column is the corresponding parcellations. CM: Corpus Medullare, Ver: Vermis, R: Right Lobule, and L: Left Lobule.

Figure 9:. Coronal slices of three testing images of the Pediatric Cohort.

The left column is the images and the right column is the corresponding parcellations. CM: Corpus Medullare, Ver: Vermis, R: Right Lobule, and L: Left Lobule.

Figure 10:. Three coronal slices of a Kirby subject.

The left column is images and the right column is the corresponding parcellations. CM: Corpus Medullare, Ver: Vermis, R : Right Lobule, and L: Left Lobule.

Figure 11:. Example parcellations of the Kwyjibo data set.

The left column is images and the right column is parcellations. ACAPULCO can parcellate cerebella with atrophy. (A): a healthy subject, (B): an SCA2 subject, (C): an SCA3 subject, and (D): an SCA6 subject.

Figure 12:. Example parcellations of the OASIS3 data set.

The left column is images and the right column is parcellations. (A): a healthy subject’s scans taken 495 days apart, (B): an AD subject’s scans taken 707 days apart.

Figure 13:. Example parcellations of the ABIDEII data set.

The left column is images and the right column is parcellations. (A): a healthy subject and (B) an ASD subject.

Figure 14:. Comparison between locating networks trained with and without translation augmentation.

The right column is the translated image with respect to the left column along the x axis. The bounding boxes are predicted by networks (A) with and (B) without translation augmentation, respectively. Note that the network on the right fails to move the bounding box accordingly without translation augmentation.

Figure 15:. Comparison between parcellating networks trained (A) with and (B) without scaling augmentation.

Note that the network on the right trained without scaling augmentation fails to label part of the cerebellum as indicated by the yellow arrows.

Figure 16:. Mislabeling the neck as part of the cerebellum.

(A): an image from the OASIS3 data set, (B): the output of the CNNs, and (C): the post-processing result. The FOVs of the training images do not cover the neck. Note that the image in (A) contains the neck and the CNNs fails to classify it as non-cerebellar in (B). Since the mislabeling is connected to the cerebellum, our post-processing cannot remove it in (C).

Figure 17:. Oversegmentation when the sinus is bright.

The left column is images and the right column is parcellations. (A) is a manually delineated image and (B) is the parcellation of another image in the ABIDEII data set. The yellow arrows point to the sinus.