Tractography dissection variability: What happens when 42 groups dissect 14 white matter bundles on the same dataset?

Abstract

White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.

Keywords: Bundle segmentation; Dissection; Fiber pathways; Tractography; White matter.

Figure 1.. Variation in white matter bundle segmentation.

Four example segmentations of the corticospinal tract (green) and arcuate fasciculus (cyan) show variability in the size, shape, densities, and connections of these reconstructed white matter pathways.

Figure 2.. Summary of teams and submissions.

Location of the teams’ affiliated lab (top). In total, 42 teams submitted 57 unique sets of bundle dissections, 28 utilized the provided deterministic streamlines, and 29 utilized probabilistic. Map icons are colored based on the set of streamlines utilized, with the same color-scheme as bar plots. Example submissions are shown for 14 pathways (bottom) along with a pie chart indicating the number of submissions for each bundle. Acronyms: see text.

Figure 3.. Variation in protocols for bundle segmentation of example pathways (CST, AF, and CC) on the same subject from the same set of whole-brain streamlines.

Eight randomly selected bundle segmentation approaches for each pathway are shown as segmented streamlines and rendered as 3D streamline density maps. Variations in size, shape, density, and connectivity are qualitatively apparent. Probabilistic streamlines are shown, see supplementary material for Deterministic submissions. Random selections generated independently for each pathway. Streamlines are colored by orientation and all density maps are windowed to the same range.

Figure 4.. Similarity and dissimilarity metrics to assess reproducibility.

Example SLF datasets are used to illustrate a range of similarity values between bundles A and B (top) and between bundles A and C (bottom). Dice overlap is a volume-based measure calculated as twice the intersection of two bundles (magenta) divided by the union (red and blue). Density correlation is calculated as the correlation coefficient between the voxel-wise streamline densities (shown as a hot-cold colormap ranging from 0 to maximum streamline density) of the two bundles being compared. Bundle adjacency is calculated by taking the average distance of disagreement (not including overlapping voxels in blue) between bundles (distances shown as hot-cold colormap). Finally, streamline Dice is taken as the intersection of common streamlines divided by the union of all streamlines in a bundle and requires input bundles to be segmented from the same set of underlying streamlines (intersection shown in figure).

Figure 5.. Corticospinal Tract (CST) inter-protocol variability.

Renderings show 25%, 50%, and 75% agreement on volume and streamlines for deterministic and probabilistic tractograms. Box-and-whisker plots of Dice overlap, density correlation, and bundle adjacency quantify inter-protocol, intra-protocol, and inter-subject variability (deterministic: red; probabilistic: blue). Each data-point in the plots is derived from the summary statistic of a single submission. Note that there were no streamlines which were common to at least 75% of the protocols.

Figure 6.. Arcuate Fasciculus (AF) inter-protocol variability.

Renderings show 25%, 50%, and 75% agreement on volume and streamlines for deterministic and probabilistic tractograms. Box-and-whisker plots of Dice overlap, density correlation, and bundle adjacency quantify inter-protocol, intra-protocol, and inter-subject variability (deterministic: red; probabilistic: blue). Note that there were no streamlines which were common to at least 75% of the protocols.

Figure 7.. Corpus callosum (CC) inter-protocol variability.

Renderings show 25%, 50%, and 75% agreement on volume and streamlines for deterministic and probabilistic tractograms. Box-and-whisker plots of Dice overlap, density correlation, and bundle adjacency quantify inter-protocol, intra-protocol, and inter-subject variability (deterministic: red; probabilistic: blue).

Figure 8.. Inter-protocol variability.

Dice overlap coefficients, density correlation, bundle adjacency, and Dice streamlines for all studied pathways. Deterministic results shown in red, probabilistic in blue.

Figure 9.

Inter-protocol variation in mean FA, weighted-FA, volume (mm3), and pathway length (mm) for all studied pathways. Note that CC volume is an order of magnitude larger than all other pathways and is shown on a 10³ mm³ scale.

Figure 10.. UMAP dimensionality reduction projected bundles onto an un-scaled 2D plane.

Object color and shape represent pathways, and object size designates deterministic/probabilistic. While variation exists within pathways and within deterministic/probabilistic streamlines, the white matter pathways generally cluster together in low dimensional space. Insets visualize data points as streamline renderings, and highlight areas where similarity and/or overlap is shown across different pathways.