Automatically tracking neurons in a moving and deforming brain

Abstract

Advances in optical neuroimaging techniques now allow neural activity to be recorded with cellular resolution in awake and behaving animals. Brain motion in these recordings pose a unique challenge. The location of individual neurons must be tracked in 3D over time to accurately extract single neuron activity traces. Recordings from small invertebrates like C. elegans are especially challenging because they undergo very large brain motion and deformation during animal movement. Here we present an automated computer vision pipeline to reliably track populations of neurons with single neuron resolution in the brain of a freely moving C. elegans undergoing large motion and deformation. 3D volumetric fluorescent images of the animal's brain are straightened, aligned and registered, and the locations of neurons in the images are found via segmentation. Each neuron is then assigned an identity using a new time-independent machine-learning approach we call Neuron Registration Vector Encoding. In this approach, non-rigid point-set registration is used to match each segmented neuron in each volume with a set of reference volumes taken from throughout the recording. The way each neuron matches with the references defines a feature vector which is clustered to assign an identity to each neuron in each volume. Finally, thin-plate spline interpolation is used to correct errors in segmentation and check consistency of assigned identities. The Neuron Registration Vector Encoding approach proposed here is uniquely well suited for tracking neurons in brains undergoing large deformations. When applied to whole-brain calcium imaging recordings in freely moving C. elegans, this analysis pipeline located 156 neurons for the duration of an 8 minute recording and consistently found more neurons more quickly than manual or semi-automated approaches.

Fig 1. Schematic of analysis pipeline to segment and track neurons through time and extract their neural activity in a deforming brain.

Neurons are labeled with calcium insensitive red fluorescent proteins, RFP, and calcium sensitive green fluorescent proteins, GCaMP. Videos of the animal’s behavior and volumetric fluorescent images of the animal’s brain serve as input to the pipeline. The algorithm detects all neurons in the head and produces tracks of the neural activity across time as the animal moves.

Fig 2. Input to the pipeline.

(A) Example images from all four video feeds from our imaging system. Both scale bars are 100μm. Fluorescent images are shown with false coloring. (B)A schematic illustrating the timings from all the devices that run in open loop in our imaging setup. The camera that collects high magnification images captures at 200Hz. The two low magnification images capture at 60Hz, and the focal plane moves up and down in a 3 Hz triangle wave. The cameras are synchronized post-hoc using light flashes and each image is assigned a timestamp on a common timeline.

Fig 3. Straightening and segmentation.

(A) Centerlines are detected from the low magnification dark field images. The centerline is shown in green and the tip of the worm’s head is indicated by a blue dot. (B) The centerline found from the low magnification image is overlaid on the high magnification RFP images. The lines normal to the centerline, shown in blue, are used to straighten the image. All scale bars are 100 μm. (C) A maximum intensity projection of the straightened volume is shown. Individual neuronal nuclei are shown (D) before and (E) after segmentation.

Fig 4. Schematic of Neuron Registration Vector Encoding.

(A) The registration between a sample volume and a single reference volume is done in several steps. I. The image is segmented into regions corresponding to each of the neurons. II. The image is represented as a Gaussian mixture, with a single Gaussian for each segmented region. The amplitude and the standard deviation of the Gaussians are derived from the brightness and the size of the segmented regions. III. Non-rigid point-set registration is then used to deform the sample points to best overlap the reference point-set. IV. Neurons from the sample and the reference point-sets are paired by minimizing distances between neurons. (B) Neuron registration vectors are constructed by assigning a feature vector v_i,t to each neuron x_i,t in a sample volume x_t by performing the registration between the sample volume and a set of 300 reference volumes, each denoted by r^k. Each registration of the neuron results in a neuron match, $v_i^k$, and the set of matches becomes the feature vector v_i,t. (C) Vectors from all neuron-times are clustered into similar groups in a two step process: Hierarchical clustering (illustrated in the figure) is performed on a subset of neurons to define clusters, each of which is given a label S_n. Then each feature vector v_i,t is assigned to a cluster based on a distance metric (not illustrated). (D) The clustering of the feature vectors shown in (C) assigns an identity to each of the neurons in every volume. This allows us to track the neurons across different volumes of the recording.

Fig 5. Example of consensus voting to correct a misidentified neuron.

In volume 735, neuron #111 is found successfully and is indicated in green. In volume 736, however, the neuron is misidentified, shown in red. During the correction phase, all other time points vote for what the position of neuron #111 should based on a thin-plate spline deformation. A sample of votes are shown (blue ‘x’). Since the initial estimate of the position is far from the majority of consensus votes, a corrected position is assigned to be the centroid of the votes weighted by image intensity. This process is repeated to correct any errors for every neuron at every time.

Fig 6. Comparison of the automated Neuron Registration Vector Encoding algorithm with manual human annotation.

A previously published 4 minute recording of calcium activity (strain AML14) was annotated by hand, [10]. (A) Spheres show position of neurons that were detected by the automated algorithm. Grey indicates a neuron detected by both the algorithm and the human. All neurons detected by the human were also detected by the algorithm (70 neurons). Red indicates neurons that were missed by the human and detected only by the algorithm (49 neurons). (B) Histogram showing number of neurons that were mismatched for a given fraction of time-volumes when comparing automated and manual approaches. Only those neurons that were consistently found by both algorithm and human were considered. An automatically identified neuron was deemed correctly matched for a given time-volume if it was paired with the correct corresponding manual neuron.

Fig 7. Calcium activity traces.

Neural activity traces from 156 neurons in the brain a C. elegans as it freely moves on an agarose plate for 8 minutes (strain AML32). The neural activity is expressed as a fold change over baseline of the ratio of GCaMP6s to RFP for each neuron. The behavior is indicated in the ethogram. On the right is the locations of all of the detected neurons (the head of the worm is towards the top of the page). The neurons that have significant correlation with reverse locomotion are indicated in red. White gaps indicate instances where neurons failed to segment. This is a newly acquired recording, different from that in Fig 6.