A probabilistic histological atlas of the human brain for MRI segmentation

Abstract

In human neuroimaging, brain atlases are essential for segmenting regions of interest (ROIs) and comparing subjects in a common coordinate frame. State-of-the-art atlases derived from histology^1-3 provide exquisite three-dimensional cytoarchitectural maps but lack probabilistic labels throughout the whole brain: that is, the likelihood of each location belonging to a given ROI. Here we present NextBrain, a probabilistic histological atlas of the whole human brain. We developed artificial intelligence-enabled methods to align roughly 10,000 histological sections from five whole brain hemispheres into three-dimensional volumes and to produce delineations for 333 ROIs on these sections. We also created a companion Bayesian tool for automatic segmentation of these ROIs in magnetic resonance imaging (MRI) scans. We showcase two applications of the atlas: segmentation of ultra-high-resolution ex vivo MRI and volumetric analysis of Alzheimer's disease using in vivo MRI. We publicly release raw and aligned data, an online visualization tool, the atlas, the segmentation tool, and ground truth delineations for a high-resolution ex vivo hemisphere used in validation. By enabling researchers worldwide to automatically analyse brain MRIs at a higher level of granularity, NextBrain holds promise to increase the specificity of findings and accelerate our quest to understand the human brain in health and disease.

Fig. 1. NextBrain workflow.

a, Photograph of formalin-fixed hemisphere (lateral view). b, High-resolution (400 μm) ex vivo MRI scan, FreeSurfer segmentation and extracted pial surface (parcellated with FreeSurfer). Left, sagittal slice of MRI. Centre, corresponding FreeSurfer segmentation. Right, 3D rendering of reconstructed and parcellated pial surface. c, Tissue slabs and blocks, before and after paraffin embedding. Left, blocked coronal slice of the cerebrum. Right, blockface photo of a cerebral block. d, Histology: coronal section of cerebrum stained with LFB (left) and H&E (right). e, Artificial-intelligence-assisted labelling of 333 ROIs on LFB. Left, cerebrum; centre, brainstem; right, cerebellum. f, Initialization of affine alignment of tissue blocks using a custom registration algorithm that minimizes overlap and gaps between blocks. g, Refinement of registration with histology and nonlinear transform^,. Reconstructed coronal slice of LFB (left), H&E (middle) and labels (right), overlaid on MRI, after nonlinear registration with artificial intelligence and robust Bayesian refinement. h, Orthogonal slices of our 3D probabilistic atlas. Left, sagittal; middle, coronal; right, axial. Each voxel is painted with a linear combination the colours of each label, multiplied by their probabilities. i, Automated Bayesian segmentation of an in vivo scan into 333 ROIs using the atlas. The atlas can also be used for segmenting ex vivo MRI and as CCF for population analyses.

Fig. 2. 3D histology reconstruction of Case 1.

a, Coronal slice of 3D reconstruction; boundaries between blocks are noticeable from uneven staining. Joint registration minimizes overlap and and gaps between blocks (this reconstructed slice comprises four different blocks). b, Accurate intermodality registration with artificial intelligence techniques. Registered MRI, LFB and H&E histology of a block, with tissue boundaries (traced on LFB) overlaid. c, Orthogonal view of reconstruction, which is smooth thanks to the Bayesian refinement and avoids gaps and overlaps thanks to the regularizer. d, Visualization of 3D landmark registration errors for Case 1. Left, visualization of landmarks. Right, histogram, mean and s.d. of error magnitude for this case, compared with our previous pipeline. Error (mean ± s.d.): 1.27 ± 0.59 mm. Error: 1.42 ± 0.72 mm. See Table 1 and Extended Data Figs. 1, 2, 3 and 4 for results on the other cases.

Fig. 3. NextBrain probabilistic atlas.

a, Comparison with whole brain atlases. Portions of the NextBrain probabilistic atlas (which has 333 ROIs), the SAMSEG atlas in FreeSurfer (13 ROIs) and the manual labels of MNI based on the Allen atlas (138 ROIs). b, Close-up of three orthogonal slices of NextBrain. The colour coding follows the convention of the Allen atlas, where the hue indicates the structure (for example, purple is thalamus, violet is hippocampus, green is amygdala) and the saturation is proportional to neuronal density. The colour of each voxel is a weighted sum of the colour corresponding to the ROIs, weighted by the corresponding probabilities at that voxel. The red lines separate ROIs on the basis of the most probable label at each voxel, thus highlighting boundaries between ROIs of similar colour; we note that the jagged boundaries are a common discretization artefact of probabilistic atlases in regions where two or more labels mix continuously: for example, the two layers of the cerebellar cortex.

Fig. 4. NextBrain segmentation of ultra-high-resolution MRI.

Automated Bayesian segmentation of publicly available ultra-high-resolution ex vivo brain MRI using the simplified version of NextBrain, and comparison with the gold standard (only available for the right hemisphere). We show two coronal, sagittal and axial slices. The MRI was resampled to 200-μm isotropic resolution for processing. As in previous figures, the segmentation uses the Allen colour map with boundaries overlaid in red. We note that the manual segmentation uses a coarser labelling protocol.

Fig. 5. Fine-grained ageing signature using NextBrain.

We report the absolute value of Spearman correlation for ROI volumes versus age derived from in vivo MRI scans. a, Ageing HCP dataset. Image resolution, 0.8-mm isotropic; N, 705; age range, 36–90 years; mean age, 59.6 years; please see main text for meaning of markers (letters). b, OpenBHB dataset, restricted to subjects with ages over 35 years to match Ageing HCP. Resolution, 1-mm isotropic; N, 431; age range, 36–86 years; mean age, 57.9 years. c, Full OpenBHB dataset. N, 3,220; age range, 6–86 years; mean age, 25.2 years; please note the different scale of the colour bar. The ROI volumes are corrected by intracranial volume (by division) and sex (by regression). Further slices are shown in Extended Data Fig. 6.

Extended Data Fig. 1. 3D reconstruction of Case 2.

The visualisation follows the same convention as in Fig. 3: (A) Coronal slice of the 3D reconstruction. (B) Registered MRI, LFB, and H&E histology of a block, with tissue boundaries (traced on LFB) overlaid. (C) Orthogonal view of reconstruction, which is smooth and avoids gaps and overlaps. (D) Visualization of 3D landmark registration errors for this specific case (left); histogram of their magnitude (right); and their mean ± standard deviation (bottom), compared with our previous pipeline (Mancini et al.).

Extended Data Fig. 2. 3D reconstruction of Case 3.

The visualisation follows the same convention as in Fig. 3: (A) Coronal slice of the 3D reconstruction. (B) Registered MRI, LFB, and H&E histology of a block, with tissue boundaries (traced on LFB) overlaid. (C) Orthogonal view of reconstruction, which is smooth and avoids gaps and overlaps. (D) Visualization of 3D landmark registration errors for this specific case (left); histogram of their magnitude (right); and their mean ± standard deviation (bottom), compared with our previous pipeline (Mancini et al.).

Extended Data Fig. 3. 3D reconstruction of Case 4.

The visualisation follows the same convention as in Fig. 3: (A) Coronal slice of the 3D reconstruction. (B) Registered MRI, LFB, and H&E histology of a block, with tissue boundaries (traced on LFB) overlaid. (C) Orthogonal view of reconstruction, which is smooth and avoids gaps and overlaps. (D) Visualization of 3D landmark registration errors for this specific case (left); histogram of their magnitude (right); and their mean ± standard deviation (bottom), compared with our previous pipeline (Mancini et al.).

Extended Data Fig. 4. 3D reconstruction of Case 5.

The visualisation follows the same convention as in Fig. 3: (A) Coronal slice of the 3D reconstruction. (B) Registered MRI, LFB, and H&E histology of a block, with tissue boundaries (traced on LFB) overlaid. (C) Orthogonal view of reconstruction, which is smooth and avoids gaps and overlaps. (D) Visualization of 3D landmark registration errors for this specific case (left); histogram of their magnitude (right); and their mean ± standard deviation (bottom), compared with our previous pipeline (Mancini et al.).

Extended Data Fig. 5. 3D landmark registration error.

Sagittal, coronal, and axial slices of the continuous maps of the 3D landmark registration error. The maps are computed from the discrete landmarks (displayed in Fig. 2d and Extended Data Figs. 1–4d) using Gaussian kernel regression with σ = 10 mm. There is no clear spatial pattern for the anatomical distribution of the error across subjects.

Extended Data Fig. 6. NextBrain superior segmentation performance with respect the Allen MNI template.

Dice scores for automated segmentations computed on the OpenBHB dataset (3,330 subjects), using the Allen MNI template and NextBrain, with FreeSurfer segmentations as reference. The scores are computed at the whole regions level, i.e., the level of granularity at which FreeSurfer segments. (A) Box plots for 11 representative ROIs. On each box, the central mark indicates the median, the edges of the box indicate the 25^th and 75^th percentiles, the whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually as ‘+’. The abbreviations for the regions are: WM = white matter of the cerebrum, CT = cortex of the cerebrum, CWM = cerebellar white matter, CCT = cerebellar cortex, TH = thalamus, CA = caudate, PU = putamen, PA = pallidum, BS = brainstem, HP = hippocampus, AM = amygdala. (B) Scatter plot of Dice (averaged across the same 11 ROIs) vs age for the Allen MNI template. There is a clear negative correlation between age and accuracy: (r = −0.274, p = 1.67 × 10⁻⁵⁶, two-sided test). (C) Scatter plot for NextBrain, whose accuracy is much more consistent across the lifespan, with almost no correlation with age (r = 0.046, p = 0.009, two-sided test).

Extended Data Fig. 7. Fine-grained ageing signature using NextBrain (additional slices).

We report the absolute value of Spearman correlation for ROI volumes vs age derived from in vivo MRI scans (additional slices). The visualisation follows the same convention as in Fig. 5: (A) Ageing HCP dataset. (B) OpenBHB dataset, restricted to ages over 35. (C) Full OpenBHB dataset.

Extended Data Fig. 8. Ageing trajectories for select ROIs in HCP dataset.

Subregions of brain structures (thalamus, hippocampus, cortex, etc) show differential ageing patterns. The red dots correspond to the ROI volumes of individual subjects, corrected by intracranial volume (by division) and sex (by regression). The blue lines represent the maximum likehood fit of a Laplace distribution with location and scale parameters parametrised by a B-spline with four control points (equally space between 30 and 95 years). The continuous blue line represents the location, whereas the dashed lines represent the 95% confidence interval (equal to three times the scale parameter in either direction of the location). Volumes of contralateral structures are averaged across left and right.