Browsing Multiple Subjects When the Atlas Adaptation Cannot Be Achieved via a Warping Strategy

Abstract

Brain mapping studies often need to identify brain structures or functional circuits into a set of individual brains. To this end, multiple atlases have been published to represent such structures based on different modalities, subject sets, and techniques. The mainstream approach to exploit these atlases consists in spatially deforming each individual data onto a given atlas using dense deformation fields, which supposes the existence of a continuous mapping between atlases and individuals. However, this continuity is not always verified, and this "iconic" approach has limits. We present in this study an alternative, complementary, "structural" approach, which consists in extracting structures from the individual data, and comparing them without deformation. A "structural atlas" is thus a collection of annotated individual data with a common structure nomenclature. It may be used to characterize structure shape variability across individuals or species, or to train machine learning systems. This study exhibits Anatomist, a powerful structural 3D visualization software dedicated to building, exploring, and editing structural atlases involving a large number of subjects. It has been developed primarily to decipher the cortical folding variability; cortical sulci vary enormously in both size and shape, and some may be missing or have various topologies, which makes iconic approaches inefficient to study them. We, therefore, had to build structural atlases for cortical sulci, and use them to train sulci identification algorithms. Anatomist can display multiple subject data in multiple views, supports all kinds of neuroimaging data, including compound structural object graphs, handles arbitrary coordinate transformation chains between data, and has multiple display features. It is designed as a programming library in both C++ and Python languages, and may be extended or used to build dedicated custom applications. Its generic design makes all the display and structural aspects used to explore the variability of the cortical folding pattern work in other applications, for instance, to browse axonal fiber bundles, deep nuclei, functional activations, or other kinds of cortical parcellations. Multimodal, multi-individual, or inter-species display is supported, and adaptations to large scale screen walls have been developed. These very original features make it a unique viewer for structural atlas browsing.

Keywords: 3D; brain atlas; inter-subject; parcellation atlas; structural approach; visualization.

FIGURE 1

Schematic representation of various configurations of a sulcus, which illustrates the absence of general point-to-point matching between subjects. The schema represents five superior frontal sulci, and some configurations can be observed in real data in Figure 5. (A) Single, long sulcus configuration. (B) Interrupted sulcus with parallel overlaps. (C) Interrupted sulcus with small branches. (D) Sulcus with a short branch and a long branch, located at a different position from configuration (C). (E) Shorter sulcus with small branches and a missing part. Superimposing the various situations here, even using non-linear warping, will not provide a perfect match.

FIGURE 2

Representation of cortical folds using Morphologist; each fold is represented by a local negative cast of the cortex, usually corresponding to the cerebrospinal fluid, which fills up the fold. (A) 2D representation of folds. (B) 3D representation as voxels lists. (C) 3D representation as meshes. The hemisphere pial mesh is also displayed with slight transparency.

FIGURE 3

An example of single subject, multimodal data visualization using Anatomist. On this example, we have displayed: (A, top left) a skull-stripped T1 MRI slice and Freesurfer segmentation overlaid, a cortical sulci graph in 3D obtained using Morphologist, a subset of a tractography reconstructed by MRTrix, a clipped gray/white matters interface mesh with Desikan cortical regions obtained by Freesurfer; (B, top right) the brain gray/white interface mesh and cortical sulci; (C, bottom left) a skull-stripped T1 MRI slice and Freesurfer segmentation overlaid in coronal orientation and a left hemisphere gray/white mesh; (D, bottom right) a skull-stripped MRI and a 2D representation of cortical sulci in axial orientation, with the gray/white mesh intersection on this plane (green). The objects live in different coordinate systems for acquisition and/or processing reasons (different kinds of software are processing using different spaces), and affine transformations between them have been loaded in Anatomist. Object’s location and cursor position match in all views.

FIGURE 4

Structural edition capabilities in Anatomist. Top: copy/paste label operation. (A) A sulcal graph (left hemisphere) and a sulci model representation are displayed in Anatomist. (B) The middle frontal sulcus is selected on the model, and its label copied; its cyan blue color appears on the lateral bar of all views. (C) A fold is selected on the subject sulcal graph, and the label is pasted on it; its color switched from yellow (intermediate pre-central sulcus) to blue (middle frontal). Bottom: fold cut and split operation. (D) An unlabeled sulcal graph is displayed; we want to split a large fold belonging to both the superior temporal sulcus, and the anterior ascending terminal branch of this sulcus. In “fold split” mode, we draw a few points materialized by red bullets (inside the red rectangle); (E) the cut operation follows a line going through selected points and splits the fold graph node into two new ones. Mesh representation is replaced with a simpler “voxels list” representation until meshing is processed again; (F) the new folds are labeled using the copy/paste features.

FIGURE 5

Structural atlas browsing, displaying the same entities on many subjects; here, 11 left hemispheres are displayed, each with 3 sulci: central sulcus (selected, in red), superior frontal (green), and inferior frontal. The latter is made up of two distinct entities, the anterior (purple) and posterior (pink) parts. The sulcal model is also displayed in the upper right 3D view. A structured display of the hierarchical nomenclature is shown on the right. All 3D views are displayed using the same orientation in a common space, although each piece of data lives in its own native coordinate system.

FIGURE 6

Multimodal structural atlas browsing; A subset of sulcal graphs and labeled fiber bundles are displayed simultaneously for three different subjects and an average model in a selected region of the brain. The top left view displays the sulci computed from the average icbm152 MRI using Morphologist and a few of the short bundles of an 80-subject atlas (Labra et al., 2019). The other views show instances of the sulcus and bundle structural atlases in 3 individual subjects, each in its individual native space. Sulcal and bundles data are displayed with nomenclatures, which are used to identify the same structures across subjects, and assign colors to them. Hemisphere brain meshes are displayed in a wireframe mode, here, to allow a better visualization of the fiber trajectories, which circumvent the folds.

FIGURE 7

A fold labeling session in front of a wall-size screen to design a chimp-based structural atlas of cortical folding.

FIGURE 8

Inter-species sulci comparison structural atlas; the first two lines of the figure display sulci of 6 different species (from left to right, and then, top to bottom: a macaque, a pongo, a chimpanzee, a baboon, a gorilla, a human). Sulci have been manually identified on these brains, using a common sulci nomenclature, in order to get structural correspondence between subjects/species. The last line extends the comparative study to extinct species using skull endocasts, using another Anatomist feature, allowing to draw sulcal lines (Le Troter et al., 2012) [from left to right: Sinanthrope XII (an Homo erectus), a recent Homo sapiens, La Ferrassie 1 (an Homo neanderthalensis] (Balzeau and Mangin, 2021).

FIGURE 9

Non-linear correspondence between different views and data. Three atlases and a sulci model representation are displayed. The ICBM152c asymmetric (top left), BigBrain (bottom left), and Colin27 (bottom right) templates are displayed, each in its native space, each with its own colored sulci set extracted using the Morphologist pipeline (in 2D here). Non-linear coordinates transformations apply between cursor positions in each view, thanks to DISCO/DARTEL transformation fields. The cursor position set by clicking on the superior frontal sulcus on the Colin27 image (bottom right) thus points to the same sulcus in the other templates’ views in different spaces, even when the atlas has large deformations such as on the postmortem BigBrain. Additionally, the sulcal model of Morphologist (top right) aligned to the ICBM152c space also displays the same sulcus at the cursor position.