Tensor image registration library: Deformable registration of stand-alone histology images to whole-brain post-mortem MRI data

Abstract

Background: Accurate registration between microscopy and MRI data is necessary for validating imaging biomarkers against neuropathology, and to disentangle complex signal dependencies in microstructural MRI. Existing registration methods often rely on serial histological sampling or significant manual input, providing limited scope to work with a large number of stand-alone histology sections. Here we present a customisable pipeline to assist the registration of stand-alone histology sections to whole-brain MRI data.

Methods: Our pipeline registers stained histology sections to whole-brain post-mortem MRI in 4 stages, with the help of two photographic intermediaries: a block face image (to undistort histology sections) and coronal brain slab photographs (to insert them into MRI space). Each registration stage is implemented as a configurable stand-alone Python script using our novel platform, Tensor Image Registration Library (TIRL), which provides flexibility for wider adaptation. We report our experience of registering 87 PLP-stained histology sections from 14 subjects and perform various experiments to assess the accuracy and robustness of each stage of the pipeline.

Results: All 87 histology sections were successfully registered to MRI. Histology-to-block registration (Stage 1) achieved 0.2-0.4 mm accuracy, better than commonly used existing methods. Block-to-slice matching (Stage 2) showed great robustness in automatically identifying and inserting small tissue blocks into whole brain slices with 0.2 mm accuracy. Simulations demonstrated sub-voxel level accuracy (0.13 mm) of the slice-to-volume registration (Stage 3) algorithm, which was observed in over 200 actual brain slice registrations, compensating 3D slice deformations up to 6.5 mm. Stage 4 combined the previous stages and generated refined pixelwise aligned multi-modal histology-MRI stacks.

Conclusions: Our open-source pipeline provides robust automation tools for registering stand-alone histology sections to MRI data with sub-voxel level precision, and the underlying framework makes it readily adaptable to a diverse range of microscopy-MRI studies.

Keywords: Brain; Histology; Human; MRI; Post-mortem; Registration.

Fig. 1

Schematic representation of a generic TIRL registration workflow. Specialised instances of this workflow are implemented by all four stages of the pipeline, employing specialised subclasses of the Cost, Optimiser and Regulariser base classes. For a detailed description of the objects/classes, the reader is referred to the general documentation of TIRL. A coarse overview of the TImage object and the workflow is given in Section 2.1 of the main text.

Fig. 2

Overview of the MRI-histology dataset for demonstrating histology-to-MRI registration with TIRL. Fourteen (11 MND + 3 control) post-mortem brains with a consistent set of multi-modal MRI data, dissection photographs, and digitised histology slides were resourced from a previous study (Pallebage-Gamarallage et al., 2018). Further details are given in Section 2.2 of the main text.

Fig. 3

Overview of the histology-to-MRI registration pipeline with two photographic intermediaries. Each stage maps the pixel coordinates of the input image to the pixel/voxel coordinates of the output image by a chain of transformations. The stage-specific transformation chains are optimised separately and eventually combined to obtain a one-to-one (invertible) mapping between histology and MRI. Due to the generality of the transformations, each histology image is mapped onto a parametric surface in MRI space. Images are not shown to scale.

Fig. 4

Stage 1 – deformable registration of a histology image to a tissue block photograph. Contrast differences between the input images are equalised by applying the non-linear image filter MIND. Image dissimilarity is defined as the Euclidean distance between the MIND representation of the images. The parameters of the Stage-1 transformation chain are found in three successive linear and one non-linear optimisation steps. See further details in Section 2.4 of the main text.

Fig. 5

Stage 2 – Automated sampling site matching and deformable registration of tissue blocks to coronal brain slabs. The raw inputs (A–F) are background-extracted (G–L), and the sampling sites on G are automatically identified by binarizing and pairwise subtracting (XOR) subsequent photographs of the coronal brain slab (G–K). Tissue block photographs (F) are cross-matched with the identified sampling sites (M) and their alignment is fine-tuned (N) at the relevant site using both linear and diffusion-regularised deformable registration.

Fig. 6

Stage 3 – Deformable registration of a brain slab photograph to an MRI volume. The four tiles from left to right illustrate consecutive steps of the optimisation process. See the main text for further details.

Fig. 7

Stage 4 pre-processing – hemisphere-specific deformable registration of a coronal brain slab photograph to a structural MRI volume. The red contour represents the grey-white matter boundary as it is seen on the brain slab photograph (A). Due to the closing of the interhemispheric fissure (green arrowhead), the bilaterally driven Stage-3 registration result (B) is not uniformly accurate (purple arrowheads). (C,D) Stage-3 registrations with hemisphere-specific masking produce accurate results. (E,F) Hemisphere-specific Stage-3 registration reveals large differences in the slicing plane between the left and the right hemispheres, which is most likely caused by the antero-posterior shearing of the hemispheres during dissection. (G) Merging hemisphere-specific slice-to-volume transformations results in a single smooth transformation of the slice that preserves the accuracy of the alignment in both hemispheres irrespective of variations in the interhemispheric gap or the antero-posterior shearing of the hemispheres (encircled).

Fig. 8

Registration error expressed as median contour distances (in mm) shown for 14 callosal and 14 hippocampal sections after the linear (white background) and non-linear (green background) steps of Stage 1. The non-linear registration error is reported for a range of different regularisation weights.

Fig. 9

Comparison of histology-to-block registration by Stage 1 and various ANTs paradigms. Top: distribution of the registration error (MCDs in mm) corresponding to the four registration paradigms tested on 14 callosal and 14 hippocampal slides. Bottom: a visual comparison of registration results on representative callosal and hippocampal sections obtained with TIRL Stage 1 and ANTs SyN CC registration. The red and blue contours represent manual segmentations of the grey-white matter boundary in the tissue block photo and the PLP-stained histology images, respectively. These and similar contours were used to compute the MCDs.

Fig. 10

Accuracy of Stage-2 registration of tissue blocks in various anatomical regions. (A) Tissue block photograph showing the left visual cortex. Grid spacing: 5 mm. (B) Left visual cortex region of the corresponding brain slab photograph shown after alignment with (A). (a,b,c): colour-coded edge-enhanced overlay of (A, red) and (B, green) within the marked regions, demonstrating the alignment of perforating vessels. The yellow colour emerges from red-green overlap, indicating accurate alignment between anatomical contours. (C-D) Registered right hippocampus block. (E-F) Registered left parahippocampal gyrus.

Fig. 11

Resilience of the Stage-2 registration algorithm against simulated block initialisation errors. The purple crosses mark the true centre of the blocks on the brain slab photographs. The green dots represent random initialisations associated with a successful (MCD < 0.2 mm) registration result, whereas red dots correspond to unsuccessful registrations. The red graphs represent 100 − #failures within a given radius from the true centre. The dashed grey lines indicate the median and the 95th percentile values of the true block initialisation error measured on the whole dataset.

Fig. 12

Stage-3 (slice-to-volume) registration result of an actual coronal brain slab photograph. The grey-white matter boundary (red contour) was segmented by hand on the brain slab photograph (A) and overlaid on the resampled MRI data after each optimisation step (B–D) to assess the accuracy of the registration. Notable misalignments are indicated by the yellow arrowheads. Through-plane deformations (D) are essential for an accurate registration of this brain slab photograph. (E) The conservative range of Jacobians suggest moderate in-plane deformations, while the 3D deformations of the slicing plane (F) are remarkable (the scale shows displacements in mm).

Fig. 13

Quantifying Stage-3 (slice-to-volume) registration error using four different sets of simulated slices. Each series (straight planar, oblique planar, straight quadratic, oblique quadratic) consists of 10 simulated slices in the postero-anterior direction. The median registration error (MRE) is plotted for each slice after each optimisation step (0: perturbed initial state, 1: rigid, 2: affine, 3: in-plane deformations, 4: 3D deformations). The gradual convergence of the MRE towards zero in all cases demonstrates the robustness of Stage 3 as well as the added value of each optimisation step.

Fig. 14

Stage 4 can improve the accuracy of histology-to-MRI registration. Top row: visual cortex, Bottom row: hippocampus. In both cases, the histology sections were sampled from deeper inside the tissue block, hence they exhibit a slightly different anatomical pattern than the corresponding tissue block photographs that were used in Stage 1. The red centre lines are provided to guide the eye. The main areas of improvement after Stage 4 are highlighted by the orange circles. Also note that tissue contours appear less distorted in the Stage 4 results, because Stage 4 deformations are defined with fewer degrees of freedom to mitigate any previously overestimated deformations of the tissue.

Fig. 15

Example of a registered MRI-histology stack of the left visual cortex, consisting of five histology stains (PLP, Iba1, CD68, SMI-312, pTDP-43), various relaxometry (T1, T2, T2*, QSM), and diffusion MRI modalities (MD, AD, RD, FA, V₁). All images are pixelwise aligned (the red centre lines are provided to guide the eye). V₁: PLP-stained histological section of the left visual cortex overlaid with a map of principal fibre orientations derived from post-mortem diffusion MRI data via diffusion tensor fitting. The fibre orientation vectors are automatically rotated by TIRL in accordance with the transformations of the histology slide.