Neuroethology of decision-making

Abstract

A neuroethological approach to decision-making considers the effect of evolutionary pressures on neural circuits mediating choice. In this view, decision systems are expected to enhance fitness with respect to the local environment, and particularly efficient solutions to specific problems should be conserved, expanded, and repurposed to solve other problems. Here, we discuss basic prerequisites for a variety of decision systems from this viewpoint. We focus on two of the best-studied and most widely represented decision problems. First, we examine patch leaving, a prototype of environmentally based switching between action patterns. Second, we consider social information seeking, a process resembling foraging with search costs. We argue that while the specific neural solutions to these problems sometimes differ across species, both the problems themselves and the algorithms instantiated by biological hardware are repeated widely throughout nature. The behavioral and mathematical study of ubiquitous decision processes like patch leaving and social information seeking thus provides a powerful new approach to uncovering the fundamental design structure of nervous systems.

Figure 1

Depending on the mechanisms present in the ancestral state, dissimilar behavioral problems can be solved by common mechanisms, and similar behavioral problems can be solved by disparate mechanisms, (a) Analogy for the relationships between algorithm, mechanism, and function. A fundamental design, the airfoil, can be configured into two very different forms: the wing and the fan. These more elaborate configurations may be further modified for use in multiple, unrelated functions, as in the use of helicopter blades for flying or fan blades for cooling. (b) We posit that the integrate-to-threshold algorithm, like the airfoil, is an evolutionary “building block” that will be found frequently in various species and implementations. This algorithm can be implemented by several mechanisms, such as neural computational models as well as gene-regulatory mechanisms. As in (a), these specific mechanisms are integrated into the evolutionary toolkit, and made available for modification. Duplication and subsequent refinement of a particular mechanism can then adapt it to subserve a different behavior. For example, a neural mechanism that originally evolved to guide foraging behavior needs relatively small modifications before it can be implemented to enhance mate-seeking behavior.

Figure 2

Rhesus macaques forage nearly optimally in a computerized patch-leaving task, and the rising value of leaving a patch is represented by single neurons in the macaque frontal cortex.(a) Monkeys remain in the patch longer as travel time rises, as predicted by the marginal value theorem (MVT). Each dot indicates a single patch-leaving decision (n = 2,834 patch-leaving events). The time at which the monkey chose to leave the patch (y axis) was defined relative to the beginning of foraging in that patch. Travel time was kept constant in a patch (x axis). Data from two monkeys are shown. Behavior (average is traced by the blue line) closely followed the rate-maximizing leaving time (red line), albeit delayed by 0–2 s. (b) Peri-stimulus time histograms (PSTHs) for an example cell in anterior cingulate cortex (ACC). Neurons responded phasically around the time of decision-making saccades and then fell to a baseline level between trials. Time zero indicates end of saccade, indicating choice. Black rectangle indicates the average duration of the trial. The firing rate during the peri-saccadic decision-making period rose with each successive decision to stay in a given patch, across multiple actions unfolding over tens of seconds. Each panel indicates responses selected from one range of patch residence times. (c) Average responses of example neuron occurring in a series of 1-s peri-saccadic epochs. Firing rates increased as time in patch increased. Error bars represent s.e.m. Firing rates peaked with the decision to abandon a patch and move on to the next. Figure after [5], used with permission.

Figure 3

Social information is a valuable resource for macaque monkeys. (a) A rhesus macaque on Cayo Santiago assumes a vulnerable posture (left) to drink from a puddle, but periodically interrupts this posture in order to visually scan the surrounding region for potential threats (right). There are no predators on the island, but aggressive social interactions are commonplace. (b) Values determined for different image classes for two male monkey subjects (open and closed bars), in ms of fluid delivery time. Positive deflections indicate the subject was willing to forgo fluid to view that image class. Negative deflections indicate the subject required fluid overpayment to choose that image class. Hindquarters refers to the perineal sexual signals of familiar females. Dominant and subordinate refer to the faces of familiar dominant and subordinate males. Gray refers to a plain gray square matched for size and luminance to the other image classes. Behavioral data depicted here corresponds to neural data depicted in Figure 4, below. Photographs by K. K. Watson.

Figure 4

The value of social information is signaled by neurons in the macaque visual orienting system. (a) Average firing rate for 34 LIP neurons plotted against time for all trials in which the subject chose to view the image (T2) in the neuron’s response field, separated by image class. (b) Average firing rate of the population for all trials in which the subject chose to view the image (T2) in the neuron’s response field, separated by fluid value relative to the non-chosen target (T1). (c) Firing rates during the 200 ms after target onset, plotted as a function of image value (left) and as a function of difference in fluid payoff between T2 and T1 (right). Regressions were performed on all data in which the subject chose to view the image. The data in (c) were binned for display, but all regressions were performed on raw data. **p < 0.001. Figure after [19], used with permission.