Auto-encoded Latent Representations of White Matter Streamlines for Quantitative Distance Analysis

Abstract

Parcellation of whole brain tractograms is a critical step to study brain white matter structures and connectivity patterns. The existing methods based on supervised classification of streamlines into predefined streamline bundle types are not designed to explore sub-bundle structures, and methods with manually designed features are expensive to compute streamline-wise similarities. To resolve these issues, we propose a novel atlas-free method that learns a latent space using a deep recurrent auto-encoder trained in an unsupervised manner. The method efficiently embeds any length of streamlines to fixed-size feature vectors, named streamline embedding, for tractogram parcellation using non-parametric clustering in the latent space. The method was evaluated on the ISMRM 2015 tractography challenge dataset with discrimination of major bundles using clustering algorithms and streamline querying based on similarity, as well as real tractograms of 102 subjects Human Connectome Project. The learnt latent streamline and bundle representations open the possibility of quantitative studies of arbitrary granularity of sub-bundle structures using generic data mining techniques.

Keywords: Diffusion MRI; Recurrent auto-encoder; Streamline clustering; Streamline embedding; Streamline tractography.

Fig. 1

LSTM-based auto-encoder architecture. The first half of the streamline was fed into the encoder of the network sequentially and the hidden state of the encoder was passed to the decoder. The decoder learned to point-by-point predict the second half of the streamline

Fig. 2

Training and validation workflow. During training, the whole brain streamlines were used to train the auto-encoder. After trained, only the encoder part was used for embedding any length of streamlines into their fixed-size latent vectors

Fig. 3

Streamline clustering of CA/CP and left/right CST, and their t-SNE 2D feature distributions. a-c Visual inspection of the clustering results for CA and CP; d the t-SNE 2D projection of the latent vectors of all streamlines in CA and CP showing a clearly separation; e–g clustering results of the left and right CST; and h the t-SNE 2D projection of the latent vectors of all streamlines for the left and right CST

Fig. 4

Streamline clustering for left/right symmetric bundles. The symmetric bundles are embedded into latent representations and clustered into two groups. UF, ILF, POPT, SLF, CST and OR are perfectly clustered, while the ICP, cingulum and SCP failed to discriminate between symmetric parts

Fig. 5

Decomposition of the left hemisphere and the clustered bundles. All the streamlines in the left hemisphere were converted into latent vectors and clustered into ten groups. Most of the streamline bundles can be extracted, while some groups contain parts from two bundles

Fig. 6

t-SNE 2D feature distributions of the ten selected bundles on the left hemisphere. The original feature vectors are 128 dimensions, and the dimensions are reduced to 2D by using t-SNE. Visual inspection of the feature distributions indicates that the streamlines from the same bundle types are closer to each other

Fig. 7

Query similar streamlines by various distances. A streamline is randomly selected from left CST and used as the seeding streamline to query similar streamlines based on various distance thresholds. Use of a smaller threshold results in sub-bundles that are very similar to the seeding streamline, while a larger threshold leads to the extraction of the entire left CST bundle

Fig. 8

Sorted embedding distances of a randomly sampled streamline

Fig. 9

Bundle-wise distance matrix. Each bundle is represented by its bundle embedding which is the average of all the streamlines in this bundle. The bundle-wise distance matrix is constructed by computed the pairwise distances among all the bundle types

Fig. 10

Hierarchical dissection of streamlines in the corpus callosum using streamline embedding clustering. On each level, the sub-bundles are further clustered into two groups iteratively. On level 3, the CC bundle can be dissected into eight sub-bundles along the anterior–posterior direction. Among the sub-bundles on level 3, the second leaf sub-bundles are identified as false streamlines

Fig. 11

Classification accuracy distributions of top-1, top-3 and top-5 measurements on all the 72 bundle types for the 101 testing subjects