Bio-physically plausible visualization of highly scattering fluorescent neocortical models for in silico experimentation

Abstract

Background: We present a visualization pipeline capable of accurate rendering of highly scattering fluorescent neocortical neuronal models. The pipeline is mainly developed to serve the computational neurobiology community. It allows the scientists to visualize the results of their virtual experiments that are performed in computer simulations, or in silico. The impact of the presented pipeline opens novel avenues for assisting the neuroscientists to build biologically accurate models of the brain. These models result from computer simulations of physical experiments that use fluorescence imaging to understand the structural and functional aspects of the brain. Due to the limited capabilities of the current visualization workflows to handle fluorescent volumetric datasets, we propose a physically-based optical model that can accurately simulate light interaction with fluorescent-tagged scattering media based on the basic principles of geometric optics and Monte Carlo path tracing. We also develop an automated and efficient framework for generating dense fluorescent tissue blocks from a neocortical column model that is composed of approximately 31000 neurons.

Results: Our pipeline is used to visualize a virtual fluorescent tissue block of 50 μm³ that is reconstructed from the somatosensory cortex of juvenile rat. The fluorescence optical model is qualitatively analyzed and validated against experimental emission spectra of different fluorescent dyes from the Alexa Fluor family.

Conclusion: We discussed a scientific visualization pipeline for creating images of synthetic neocortical neuronal models that are tagged virtually with fluorescent labels on a physically-plausible basis. The pipeline is applied to analyze and validate simulation data generated from neuroscientific in silico experiments.

Keywords: Fluorescence rendering and visualization; Highly scattering volumes; In silico neuroscience; Modeling and simulation; Neocortical brain models.

Fig. 1

Light transport in a highly scattering volumetric extent. a The volume prior to illumination by the light source. b Single scattering interaction: the light ray is scattered once between the light source and the camera on a single path x ₀ x ₁ x ₂. c Multiple scattering: the light ray bounces multiple times between several interaction events before reaching the camera on a single path x ₀ x ₁ x ₂…x _n−1 x _n. d The radiative transport equation evaluates the light propagating from the light source to the camera on multiple paths ${\overline{x}}^1,{\overline{x}}^2,\dots ,{\overline{x}}^n$. The rays are shot from the camera towards the light source to sample the scattering events

Fig. 2

Path tracing with multiple scattering in fluorescent volume. The green and yellow rays are transported at λ _m and λ _x respectively. The red rays escape the volume with no contribution to the estimated radiance along the path. The dashed rays indicate invalid paths, where fluorescence visibility is zero. The light is only sampled if a fluorescence emission event is determined

Fig. 3

All possible combinations of interaction events during path sampling in a scattering fluorescent mixture. The white/green events represent an interaction between the light ray and non-fluorescent/fluorescent volume samples. The events in (a) and (b) are not physically-plausible because a fluorescent emission must occur at a fluorescent sample. f is also not possible because λ _m cannot excite the dyes to emit at λ _x. The events in (c), (d), (g) and (h) represent an elastic scattering at the same wavelength. e is the only event that can account for fluorescence emission

Fig. 4

The process of creating a fluorescent tissue block from the cortical column model. a The meshes of each neuron in the column are created and loaded according to their position and orientation specified by a given micro-circuit configuration. b The requested mesh block is extracted from the neocortical column model in (a). c The mesh block is converted into a volume with solid voxelization. d The volume block is annotated with the optical properties of the brain and the spectroscopic properties of the dyes specified in the input configuration file. The density of the cells in the illustrated model in A is only 5%

Fig. 5

An illustration of the mesh block extraction from the selected targets in the cortical column. a The spatial extent of the block is identified by a bounding box that is given in the input configuration. b The meshes are generated from the corresponding morphologies with an extended version of the meshing pipeline presented by Lasserre et al. [44]. c The resulting wavefront object meshes are loaded in Blender [45] and clipped on a per-mesh basis. d All the clipped meshes are loaded in Blender and grouped together with a union boolean operation to generate the final mesh block

Fig. 6

Surface rendering of a watertight mesh of a 50 μm³ tissue block extracted from a digital reconstruction of the microcircuitry of the somatosensory cortex of a two-week-old rat. The model is textured with an electron microscopy shader and loaded in Maya (Autodesk, California, USA) [59] for rendering

Fig. 7

Volume rendering of a 50 μ m ³ fluorescent neuronal model block tagged in silico with three Alexa Fluor 488 solutions that are characterized by low (left), medium (middle) and high (right) extinction coefficients. The volumes are illuminated with monochromatic diffusive light source that emits at 495 nm corresponding to the maximum excitation wavelength of the Alexa Fluor 488 dye

Fig. 8

Volume rendering of the tissue volume blocks when the neurons are virtually injected with four different fluorescent dyes: a Alexa Fluor 350, b Alexa Fluor 488, c Alexa Fluor 586 and d Alexa Fluor 633. The volumes are illuminated with monochromatic laser sources at 346, 495, 578 and 632 nm that correspond to the maximum excitation wavelength of the four fluorescent dyes respectively

Fig. 9

Normalized emission SPDs measured from the images illustrated in Fig. 8. The spectral responses of the emission recorded from each tissue block is qualitatively compared with the actual emission spectra of the four Alexa Fluor dyes used to tag the tissue block. The SPDs are obtained at the maximum excitation wavelengths of each respective dye (346, 495, 578 and 632 nm) and 1024 spectral samples per pixel

Fig. 10

Relative emission SPDs measured from the images generated from rendering the four fluorescent tissue blocks tagged with Alexa Fluor 350, – 488, – 568 and – 633 at different excitation wavelengths between 300 and 700 nm. The profiles are normalized to the SPD measured at maximum excitation wavelength for each respective dye. The SPDs are detected at 1024 spectral samples per pixel. Notice the relation between the amplitude of the excitation spectrum of each dye at the exciting wavelength and maximum amplitude of measured SPD

Fig. 11

A high level overview of the in silico experimentation workflow. The scientists extract a tissue block from the neocortical column model, tag it virtually with a specific fluorescent dye and use it in in silico fluorescent-based experiment. The renderings are analyzed and validated, and the tissue model is improved