Automatic 3D Nonlinear Registration of Mass Spectrometry Imaging and Magnetic Resonance Imaging Data

Abstract

Multimodal integration between mass spectrometry imaging (MSI) and radiology-established modalities such as magnetic resonance imaging (MRI) would allow the investigations of key questions in complex biological systems such as the central nervous system. Such integration would provide complementary multiscale data to bridge the gap between molecular and anatomical phenotypes, potentially revealing new insights into molecular mechanisms underlying anatomical pathologies presented on MRI. Automatic coregistration between 3D MSI/MRI is a computationally challenging process due to dimensional complexity, MSI data sparsity, lack of direct spatial-correspondences, and nonlinear tissue deformation. Here, we present a new computational approach based on stochastic neighbor embedding to nonlinearly align 3D MSI to MRI data, identify and reconstruct biologically relevant molecular patterns in 3D, and fuse the MSI datacube to the MRI space. We demonstrate our method using multimodal high-spectral resolution matrix-assisted laser desorption ionization (MALDI) 9.4 T MSI and 7 T in vivo MRI data, acquired from a patient-derived, xenograft mouse brain model of glioblastoma following administration of the EGFR inhibitor drug of Erlotinib. Results show the distribution of some identified molecular ions of the EGFR inhibitor erlotinib, a phosphatidylcholine lipid, and cholesterol, which were reconstructed in 3D and mapped to the MRI space. The registration quality was evaluated on two normal mouse brains using the Dice coefficient for the regions of brainstem, hippocampus, and cortex. The method is generic and can therefore be applied to hyperspectral images from different mass spectrometers and integrated with other established in vivo imaging modalities such as computed tomography (CT) and positron emission tomography (PET).

Figure 1.

Proposed framework for automatic integration of 3D MSI and MRI: a) 3D MSI datset consists of a set of sequential 2D MSI datacubes each of which holds hundreds of m/z ion images. b) t-SNE images simplify the dimensional complexity of each 2D MSI datacube in a single image representation. The color of the t-SNE images is arbitrarily chosen to visualize spectrally differentiated regions that created edge structures to guide the multi-modal registration. The 3D MSI-MRI image registration was implemented in a slice-to-slice fashion between the t-SNE and MR images as illustarted in b and c. Panels (d-f) shows simple visualization of inter-slice deformations of the MSI tissue sections before and after non-linear registration with their corresponding MR slices. MRI provides a volumetric reference to preserve the original tissue shape while non-linearly warping the 3D MSI data. The binary representation is for illustrative purposes, but the registration process took image contents in considerations.

Figure 2.

Multi-modal integration of MSI and MRI data of normal mouse brain: 3D distribution of two ion features at m/z 864.5 ± 0.1 and m/z 840.5 ± 0.1 that were non-linearly deformed and overlaid atop of T2-RARE MR volumetric image. The upper row shows the distribution of m/z 864.5 is highly expressed and co-localized with the MRI anatomical regions of Corpus Callosum and Brainstem. Contrary, the bottom row shows high expression of m/z 840.5 in the Cortex and Hippocampus. The volume rendering of these two ions, in the right column, reveals natural curvatures that resemble the original tissue shape.

Figure 3.

Multi-modal integration of MSI and MRI data of PDX glioblastoma mouse brain model GBM39: (a) non-linear 3D reconstruction of ion feature at m/z 394.1757 (erlotinib) that are highly expressed and localized in the tumor region. This ion feature was volume rendered and overlaid atop of T2-RARE MR volumetric image (b). Coronal cross sections show the original anatomical structures of T2-RARE MR image within the region of interest before (c) and after (d) fusing with drug molecule.

Figure 4.

Hierarchical stochastic neighbor embedding (HSNE) analysis of 3D MSI data from normal brain reveals multi-scale spectral patterns that were fused to the MR volumetric image to visualize the registration quality. (a) The overview embedding separates tissue foreground from various background noise, and the detailed embedding only on the tissue structure reveals distinct molecular patterns that agree with MR anatomical structures (c). The HSNE molecular patterns were volume rendered (b) and fused atop of T2-RARE MR image (d) and the zoomed-in images visualize high registration quality.

Figure 5.

Hierarchical stochastic neighbor embedding (HSNE) identifies spectral patterns associated with tumor and normal tissue types in the GBM39 mouse brain model. The overview embedding shows the major identified structures, whereas a more detailed embedding at level L2 depicts finer segmented structures of both tumor and normal tissues. Spectral-based segmentation of tumor (red) and normal (green) structures using HSNE were volumetric rendered (b) and overlaid atop of T2-RARE MRI (c) and (d).

Figure 6.

Evaluation of the registration quality using the Dice metric in three main anatomical regions for two healthy and untreated mouse brains.