Quantifying behavior to solve sensorimotor transformations: advances from worms and flies

Abstract

The development of new computational tools has recently opened up the study of natural behaviors at a precision that was previously unachievable. These tools permit a highly quantitative analysis of behavioral dynamics at timescales that are well matched to the timescales of neural activity. Here we examine how combining these methods with established techniques for estimating an animal's sensory experience presents exciting new opportunities for dissecting the sensorimotor transformations performed by the nervous system. We focus this review primarily on examples from Caenorhabditis elegans and Drosophila melanogaster-for these model systems, computational approaches to characterize behavior, in combination with unparalleled genetic tools for neural activation, silencing, and recording, have already proven instrumental for illuminating underlying neural mechanisms.

Figure 1. Mapping full sensorimotor pathways

Solving the sensorimotor transformations that nervous systems perform requires quantification of sensory inputs, neural dynamics and behavioral outputs. Sensory inputs influence neural activity which drives behavior, which in turn can change the sensory input that an animal receives (image of fly adapted from Muijres et al 2014).

Figure 2. Automated algorithms for quantifying behavior

(A) A fully supervised algorithm for segmenting acoustic behavior begins by (i) taking raw audio recordings, (ii) using an algorithm to identify salient features - here, ‘pulse’ and ‘sine’ types of song - and then inferring (iii) the longer-term bout structure (consisting of alternating trains of pulse and sine song) (modified from Arthur et al. 2013). (B) Starting with video data (i), other supervised algorithms (ii) find salient features such as center of mass (position), velocity, trajectory, and so on. (iii) A semi-supervised machine learning algorithm (JAABA) can use these features to identify discrete actions defined by the experimenter (modified from Kabra et al. 2013). (C) Largely unsupervised methods attempt to identify all behaviors from raw video data (i). (ii) One such method takes a set of aligned images from movies of flies and decomposes the dynamics into a low-dimensional basis set. Time series are produced by projecting the original pixel values onto this basis set, and these trajectories are then embedded into two dimensions (using t-SNE). (iii) Each position in the behavioral map corresponds to a unique set of postural dynamics, with nearby points representing similar motions (modified from Berman et al 2014). Maps are built by computing the probability of being embedded in this point in 2D space (left), then clustered with a watershed algorithm into discrete actions (middle) before identifying what general behaviors large regions of space belong to (right).

Figure 3. Inferring sensorimotor transformations from behavioral data

(A) Visual and olfactory pathways can be driven by two different colors of light use Gaussian white noise patterns (i). On the basis of this activation, Drosophila larvae will decide whether or not to turn (ii). (iii) A reverse-correlation analysis finds the filters that transform sensory neuron activation into behavioral choices (modified from Gepner et al 2015). (B) (i) C. elegans that are on a food patch will explore (insert) and experience changes in food concentration. (ii) When they are exploring off-food, they will explore a small area by emitting a certain number of large-angle turns (represented here as black dots) to interrupt their forward locomotion and keep them in a small area. (iii) This sensory experience and turning output can be linked by a linear filter that keeps track of ~25 minutes of experience (modified from Calhoun et al 2015). (C) (i) Drosophila males will court a female by observing features such as the distance between the male and female and her velocity and (ii) will produce a dynamic song via wing vibration. (iii) The type of song that is emitted can be predicted based on the male’s own velocity and the inter-fly distance, for instance (modified from Coen et al 2014).