Streamline tractography of the fetal brain in utero with machine learning

Abstract

Diffusion-weighted magnetic resonance imaging (dMRI) is the only non-invasive tool for studying white matter tracts and structural connectivity of the brain. These assessments rely heavily on tractography techniques, which reconstruct virtual streamlines representing white matter fibers. Much effort has been devoted to improving tractography methodology for adult brains, while tractography of the fetal brain has been largely neglected. Fetal tractography faces unique difficulties due to low dMRI signal quality, immature and rapidly developing brain structures, and paucity of reference data. To address these challenges, this work presents a machine learning model, based on a deep neural network, for fetal tractography. The model input consists of five different sources of information: (1) Voxel-wise fiber orientation, inferred from a diffusion tensor fit to the dMRI signal; (2) Directions of recent propagation steps; (3) Global spatial information, encoded as normalized distances to keypoints in the brain cortex; (4) Tissue segmentation information; and (5) Prior information about the expected local fiber orientations supplied with an atlas. In order to mitigate the local tensor estimation error, a large spatial context around the current point in the diffusion tensor image is encoded using convolutional and attention neural network modules. Moreover, the diffusion tensor information at a hypothetical next point is included in the model input. Filtering rules based on anatomically constrained tractography are applied to prune implausible streamlines. We trained the model on manually-refined whole-brain fetal tractograms and validated the trained model on an independent set of 11 test subjects with gestational ages between 23 and 36 weeks. Results show that our proposed method achieves superior performance across all evaluated tracts. Qualitative assessments on independent data from the Developing Human Connectome Project demonstrated the generalizability of our method. The new method can significantly advance the capabilities of dMRI for studying normal and abnormal brain development in utero.

Keywords: developing brain; diffusion MRI; fetal brain; machine learning; tractography.

Fig. 1.

Color fractional anisotropy images for fetal brains scanned at 24, 28, 32, and 36 gestational weeks. The yellow arrows point to example regions with noisy or possibly erroneous values.

Fig. 2.

The proposed fetal tractography method. The method encodes the information in the 3D volume of diffusion orientation distribution using transformer and convolutional blocks, generating feature maps at three different scales. The spatial location of the current streamline propagation point is used to interpolate these feature maps. A similar procedure is followed to encode the information in the tissue segmentation map and the fixel atlas image registered to the subject, although using a light-weight fully-convolutional network. These are combined with prior streamline propagation directions and with “position vector” features that represent the global location of the current point in the brain. A set of fully-connected layers fuse these features to predict the next streamline propagation direction. The numbers specified after the # sign on each path indicate the dimensionality of the feature map on that path.

Fig. 3.

Our proposed scheme for encoding the position of the current tractography propagation point in the brain. We encode this information as the normalized distance with respect to the centers of mass of five non-coplanar cortical parcellation regions. This figure shows the axial, sagittal, and coronal views depicting the cortical parcellation regions that are visible in the shown slices, spheres denoting the centers of mass of those regions, and 3D views that show the lines connecting these centers to an arbitrary point within the brain mask. The fetuses shown in this figure are 24 and 31 weeks of gestational age.

Fig. 4.

Axial, coronal, and sagittal views of the spatiotemporal fixel atlas for 25, 30, and 35 gestational weeks. In order to better view the atlases for lower gestational ages, all atlases have been displayed to the same size.

Fig. 5.

We launch a streamline from the center of each gray matter voxel that has at least one white matter voxel neighbor. The direction of the first step is selected to be the direction of the line connecting the center of the gray matter voxel to the center of the neighboring white matter voxel. In this figure, the red voxels are gray matter and the green voxels are white matter. We have selected three arbitrary gray matter voxels and have shown the direction of the first step for the streamlines launched from those seed points with yellow arrows. To enhance the tractogram diversity, random jittering is applied to the location of the seed point and the direction of the first step as explained in the text. These augmentations are not portrayed in this figure.

Fig. 6.

Summary of quantitative evaluation metrics for our proposed method and the compared techniques. The values shown in these plots have been pooled across 11 test subjects and 9 different white matter tracts.

Fig. 6.

Summary of quantitative evaluation metrics for our proposed method and the compared techniques. The values shown in these plots have been pooled across 11 test subjects and 9 different white matter tracts.

Fig. 6.

Summary of quantitative evaluation metrics for our proposed method and the compared techniques. The values shown in these plots have been pooled across 11 test subjects and 9 different white matter tracts.

Fig. 7.

Performance of our proposed method and compared techniques in terms of the Dice score for individual white matter tracts.

Fig. 8.

A summary of the quality scores assigned by an expert to the tracts reconstructed by the proposed method and the compared techniques. The results shown in this figure have been pooled across the 9 different tracts and the 11 test subjects. Descriptions of the scores are presented inTable 1.

Fig. 9.

Detailed tract-wise reconstruction quality scores for the proposed method and the compared techniques. Descriptions of the scores are presented inTable 1.

Fig. 10.

Visual comparison of whole-brain tractographies generated by our proposed method and other methods for a test fetus at 26 gestational weeks.

Fig. 11.

Comparison of tract reconstructions between our proposed technique and other methods on a 32-week fetus. In this example, our method reliably reconstructed all tracts, whereas other methods only managed to reconstruct a few tracts with low quality. Of note, for this specific test sample, RNN failed to reconstruct any of the tracts.

Fig. 12.

Example test results generated with our method and the reference method that was used to generate the streamlines on training images. Yellow arrows point to examples of incomplete tracts and spurious (false positive) streamlines. As shown in these examples, our method quite often performs better than the reference method and computes tracts that are more complete and have fewer false positives. In some cases, such as the top right example in this figure, the reference method completely failed to reconstruct the tract, while our method successfully reconstructed the same tract.

Fig. 13.

This figure shows the variability in the results computed by the proposed method. Each row in the figure shows a specific tract reconstructed by the proposed method for four different test subjects. These examples show that the proposed method can effectively learn, preserve, and reflect the inter-specific variability. The number at the lower left of each figure indicates the gestational age in weeks.

Fig. 14.

Tractography results computed by the proposed method on three example fetal subjects from the DHCP data. Each row presents a different subject. From left, the first three columns show the sagittal, axial, and coronal views of the whole-brain tractogram. The two right-most columns show example tracts extracted automatically with WMQL from each tractogram. The gestational ages of these fetuses from top to bottom are 27, 32, and 35 weeks.