NeuRegenerate: A Framework for Visualizing Neurodegeneration

Abstract

Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections over time is limited to observations gathered using population analysis. In this article, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject across specified age-timepoints. To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (GAN) that translates features of neuronal structures across age-timepoints for large brain microscopy volumes. We improve the reconstruction quality of the predicted neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss. Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. Finally, to visualize the change in projections, predicted using neuReGANerator, NeuRegenerate offers two modes: (i) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, from two age domains (using structural view and bounded view), and (ii) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes within the cholinergic system of the mouse brain between a young and old specimen.

Fig. 1.

The axonal projections of (a) a 6-week, and (b) a 9-month mouse, imaged using WFM. We can observe that they appear connected and healthy at 6-weeks, and fragmented and thin at 9-months (red arrows).

Fig. 2.

NeuRegenerate framework for (a) training and (b) visualization. By providing a collection of region-of-interest WFM brain volumes, neuReGANerator learns the structural features that are translated between the specified age timepoints. Once trained, neuReGANerator predicts fibers for the respective age translation. We provide 2 visualization modes: (i) neuroCompare: simultaneous visualization of predicted result and input data (using structural view and bounded view), and (ii) neuroMorph: interactive transformation of input volume to predicted result using our morphing technique.

Fig. 3.

Our spatial-consistency module improves the reconstruction quality of the output volume. For an input raw WFM volume (a), the results without and with spatial-consistency are shown in (b) and (c), respectively. We can observe that in our results (c), tiling artifacts are reduced and the intensity values of foreground and background voxels across tiles are smoother and consistent, as pointed out by the yellow arrows. Moreover, the reconstruction of neurites close to the tile borders are improved, as pointed out by the blue arrows.

Fig. 4.

(a) Training a GAN. (b) Our spatial-consistency module, with simultaneous input of 4 overlapping tiles into the generator. The center is constructed using the centers of each output from the generator. Using our spatial-consistency, the model is trained to consider information contained in a tile local neighborhood.

Fig. 5.

(i) Screenshot of our NeuRegenerate visualization application, and (ii) example of our structural visualization. On loading WFM volume, the user first defines input parameters (a). Next, the input and output age timepoints for the trained neuReGANerator are specified (b). User selects one of two visualization modes (c). User can visualize WF volumes using direct volume rendering and 1D TFs for each age domain (d). The structural visualization of the volume loaded in (i) is shown in (e) and its predicted 6-week neurites is shown in (f). This mode extracts neuronal structures from each domain and renders them in a single view, as shown in (g).

Fig. 6.

Example of our bounded view visualization. For each direction, a volume rendering of the bounded raw input WF data is shown in (a), followed by the surface rendering of its predicted structure in (b). Parameters such as intensity thersholding and gamma correction can be adjusted for the volume, as demonstrated in (c).

Fig. 7.

The path for dynamic voxels follow either (a) growing/shrinking or (b) splitting/merging action.

Fig. 8.

NeuRegenerate aids neuroscientists in visualizing structural changes that occur within a specimen brain, across age: for a diseased mouse data (left-most structure), we are able to predict and reconstruct its healthy neuronal fibers at a younger age (right-most structure). The two in-between structures are generated using neuroMorph, that allows users to interactively visualize the neurodegeneration process.

Fig. 9.

The results of neuReGANerator using 6-week and 9-month cholinergic neurons in the medial septum of a mouse brain. We focus on two regions (Crop A and B) of dimensions 500×500×20. The green volume in (i) is a testing 6-week input. The red volumes in (a) are the corresponding 9-month predicted reconstructions of the cropped regions. The red volume in (ii) is a testing 9 month-old input. Similarly, the green volumes in (b) are the corresponding 6 week-old predicted reconstructions. The plots at the bottom are histograms of the respective input volumes. The histograms demonstrate the skewness of the voxel intensity distribution, thus making visualization of WFM volumes a challenge.

Fig. 10.

The structural view visualizations for the cropped regions (Crop A and Crop B) shown in Fig. 9

Fig. 11.

Our neuroMorph visualization for the 9 months to 6 weeks Crop B example.

Fig. 12.

The structural view of the cropped cortical region.