Computational Neuroethology: A Call to Action

Abstract

The brain is worthy of study because it is in charge of behavior. A flurry of recent technical advances in measuring and quantifying naturalistic behaviors provide an important opportunity for advancing brain science. However, the problem of understanding unrestrained behavior in the context of neural recordings and manipulations remains unsolved, and developing approaches to addressing this challenge is critical. Here we discuss considerations in computational neuroethology-the science of quantifying naturalistic behaviors for understanding the brain-and propose strategies to evaluate progress. We point to open questions that require resolution and call upon the broader systems neuroscience community to further develop and leverage measures of naturalistic, unrestrained behavior, which will enable us to more effectively probe the richness and complexity of the brain.

Fig 1.. Challenges in computational ethology.

A characteristic sequence of behaviors exhibited by a mouse in its home cage as it thigmotaxis around the walls, approaches some food, eats and then sleeps (center, mouse cartoons). Several key challenges face any segmentation of continuous behavior into components are illustrated. First, the behavior of most model organisms occurs in three dimensions (green). Second, behaviors need to be labeled, which raises the problem of “lumping” versus “splitting” (red); for example, mice thigmotax, a behavior in which mice exhibit locomotion and turning behaviors that are deterministically sequenced to generate an action where the animal circumnavigates its cage. Is thigmotaxis a singular behavior (because its elements are deterministically linked during its expression), or it is a sequences of walking, turning and walking behaviors? Third, should behavior be considered at a single timescale that serially progressed, or instead considered a hierarchical process organized at multiple timescales simultaneously (blue). Fourth, when the mouse is sniffing and running at the same time, is that a compositional behavior whose basis set includes “run” and “sniff,” or is “running+sniffing” a fundamentally new behavior (purple)?

Fig. 2.. Typical stages in a behavioral analysis pipeline.

Conceptual steps involved in any behavior analysis pipeline (left) with references to published example pipelines that are openly available for use (right). Shaded region indicates the steps that each method implements. a. Animal behavior is recorded. b. Features are extracted. Features may be interpretable, e.g. a mouse’s paw position; or abstract e.g. an algorithmically defined weighted set of pixels. Automatic feature extraction can roughly be divided into algorithms that use classical computer vision feature detection, supervised learning, or unsupervised learning. High-dimensional descriptions of behavior features can optionally be approximated with fewer-dimensions via dimensionality reduction. c. To leap from features to behavior requires first representing the temporal dynamics of the features. Feature dynamics can be represented in either the time- or frequency domain. Additional dimensionality reduction can optionally be performed at this stage. d. The resulting temporal dynamics are organized into behavior which can be either discrete or continuous. For discrete behavior representations, feature dynamics are clustered and labeled (e.g. ‘Sniffing’ or ‘Turning’) using either supervised or unsupervised machine learning. For continuous descriptions of behavior, trajectories through behavior space are analyzed and interpreted e.g. using dynamical systems models.

Fig. 3.. Pose estimation.

Recent advances in artificial neural networks allow identifiable points on an animal’s surface (e.g., joints) to be automatically detected from images with minimal human supervision, even when animals interact or are in rich and complex backgrounds (Graving et al., 2019; Mathis et al., 2018; Pereira et al., 2019). Examples of a. flies, b. giraffes and c. mice are shown from (Pereira et al., 2019 and unpublished).