Whole-brain tissue mapping toolkit using large-scale highly multiplexed immunofluorescence imaging and deep neural networks

Abstract

Mapping biological processes in brain tissues requires piecing together numerous histological observations of multiple tissue samples. We present a direct method that generates readouts for a comprehensive panel of biomarkers from serial whole-brain slices, characterizing all major brain cell types, at scales ranging from subcellular compartments, individual cells, local multi-cellular niches, to whole-brain regions from each slice. We use iterative cycles of optimized 10-plex immunostaining with 10-color epifluorescence imaging to accumulate highly enriched image datasets from individual whole-brain slices, from which seamless signal-corrected mosaics are reconstructed. Specific fluorescent signals of interest are isolated computationally, rejecting autofluorescence, imaging noise, cross-channel bleed-through, and cross-labeling. Reliable large-scale cell detection and segmentation are achieved using deep neural networks. Cell phenotyping is performed by analyzing unique biomarker combinations over appropriate subcellular compartments. This approach can accelerate pre-clinical drug evaluation and system-level brain histology studies by simultaneously profiling multiple biological processes in their native anatomical context.

Fig. 1. Overview of the whole brain tissue phenotyping pipeline for processing highly multiplexed immunohistological (MP-IHC) image datasets acquired using a customized wide-field multispectral epifluorescence imaging platform (refer to Fig. 2 for details) in conjunction with integrated open source computational modules for image reconstruction, optimization and quantitative deep machine learning data analyses (refer to Figs. 3–6 for details).

a The pipeline input consists of raw high-content 2D multispectral MP-IHC imaging datasets sourced from batch slide scans of 10 μm thick serial rat brain tissue sections iteratively probed with a myriad of biomarkers, which are then computationally processed for image registration, and intra- and inter-channel correction prior to deep learning-based quantitative analyses for cell phenotyping, cell counting, and anatomical mapping. b Additional modules include 3D reconstruction from MP-IHC image datasets sourced from multiple batches of serial whole rat brain tissue sections imaged in 2D and processed for volumetric 3D data analyses.

Fig. 2. Overview of the multiplex IHC staining and multispectral epifluorescence imaging platform.

a The imaging platform optimally utilizes the commercial availability of a wide selection of spectrally compatible fluorophores across the full usable light spectrum ranging from ultraviolet to near-infrared (350–800 nm). Components of the wide-field epifluorescence microscope required for 10-color multispectral imaging include a Zeiss AxioImager.Z2 upright fluorescence microscope (left) equipped with a high-resolution objective, a high sensitivity digital camera, a broad-spectrum light excitation source and 10 self-contained excitation/dichroic/emission filter sets optimized to detect up to 10 commonly used or custom-derived spectrally compatible fluorescent reporters (Supplementary Table 2), with minimal spectral cross-talk (Supplementary Table 4), as specified in the excitation/dichroic/emission filter table (right), and exemplified in the included representative images (bottom) of major brain cell types as visualized using a 10-plex immunostaining protocol outlined in Supplementary Table 5. b The basic 10-color fluorescence imaging platform can be user-configured to include different combinations of customized filters to accommodate optimal imaging of different selections of commonly applied and custom-derived fluorescent dyes (Supplementary Table 2) and used iteratively with different selection of immunocompatible biomarkers to label a myriad of brain cell types exemplified in the included representative images in the 50-plex image dataset as visualized using a 5-round immunostaining protocol outlined in Supplementary Table 6.

Fig. 3. Overview of the post-acquisition image optimization module.

a Pixel-to-pixel registration of the separate imaging rounds by detecting the nuclei landmarks from the DAPI channel in each round (top) and applying affine transformation to register the images to the target round (bottom). b Intra-channel correction of non-specific signals introduced during imaging including uneven illumination (left), photobleaching (center), and tissue folds (right) using Alternating Sequential Filters (ASF). The registered image (top) is subtracted from the estimated intra-channel non-specific signal to get the corrected signal (bottom). c Inter-channel correction of non-specific signal to extract spectral bleed-through (left) and molecular co-localization (right) utilizing a semi-supervised sparse linear spectral unmixing approach. Original signal (top) is subtracted from the estimated mixed channel (middle) to get the specific fluorescent signal of interest (bottom).

Fig. 4. Improved nuclei detection in whole brain images using a multiplexed approach.

a Montage showing variability in DAPI (pan-nuclear DNA stain) and histone (pan-nuclear protein) expression labeling across the whole rat brain tissue slice. b–e The sum of the DAPI and pan-histone markers capture nuclear morphology more reliably than either marker alone. f–i Close-up images showing DAPI and pan-histone labeling variations in a sparse and a densely packed region and the generated location (bounding box) results of the proposed model for reliable detection of the cell nuclei using transfer learning approach in conjunction with a Faster-RCNN network. j Receiver Operating Characteristic (ROC) curve of cell nuclei detection using single marker and combination of markers shows significant improvement in performance when both markers are included.

Fig. 5. Improved methods for cell-type classification and quantification using deep morphological features compared to traditional intensity levels thresholding.

a Architecture of the proposed Capsule Network (CapsNet) to extract more comprehensive features for more accurate cell classification. b Pseudo-colored multichannel montage image of 5 major brain cell types. c Computational reconstruction of major cell-type montage from classified seeds (centers of bounding boxes) pseudo-colored to match the color of the actual biomarker expression recapitulating the specific cellular distribution of each cell type in the original raw image in (b) with high fidelity. Histogram of lengths of capsule vectors in the last layer of the network (right) compared to traditional phenotyping approach by thresholding on the histogram of mean signal intensity of the major brain cell-type classification biomarkers (left) measured inside individual bounding boxes using DAPI+histone for seed detection (depicted in Fig. 4f-i) identifying (d) neurons (NeuN), (e) astrocytes (S100β), (f) oligodendrocytes (Olig2), (g) endothelial cells (RECA1), and (h) microglia (Iba1). The histograms of the proposed method shows bimodal distribution with well-separated peaks for better separation of negative and positive population of cells for each cell phenotype with enlarged regions of interest from insets with single biomarker channels and overlaid classified seeds (white dots) illustrates the complete match of the generated cell phenotypes in the raw images.

Fig. 6. Summary of quantification and morphological modules.

a Pseudo-colored multichannel montage of the whole rat brain image scan (left) showing biomarkers used for morphological masking. b–f Selected close-up regions illustrate the original image (top), nucleus/soma mask (middle) and whole cell mask (bottom) including nucleus, cytoplasm, cell processes, plasma membrane for each major brain cell type including oligodendrocytes (b), neurons (c), astrocytes (d), microglia (e) and endothelial cells (f). g Summarized table of enumeration and percentage of cell types and subtypes in a single whole-brain 2D tissue slice shown in (a). h Pseudo-colored multichannel montage image of neuronal cell-subtype classification biomarkers including glutamatergic (GLUT), GABAergic (GAD67), cholinergic (CHAT), catecholaminergic (TH), and uncharacterized neurons. i Computational reconstruction of neuronal cell-subtype montage from classified seeds pseudo-colored to match the color of the actual biomarker expression recapitulating the specific cellular distribution in the original raw image with high fidelity. j–m Enlarged regions of interest from insets in panel (h) with single biomarker channels and overlaid classified seeds (white dots) illustrates the complete match of the generated cell phenotypes in the raw images. n Manually-fitted Paxinos atlas on the 2D brain section to quantify the number of the cells per unit area for phenotypic analysis of cells in defined anatomical regions. Heatmap plot showing the number of positive cells per 10⁶ pixel² area, exemplified for neurons (o) and the oligodendrocyte (p) population.