On the challenges and mechanisms of embodied decisions

Abstract

Neurophysiological studies of decision-making have focused primarily on elucidating the mechanisms of classic economic decisions, for which the relevant variables are the values of expected outcomes and action is simply the means of reporting the selected choice. By contrast, here we focus on the particular challenges of embodied decision-making faced by animals interacting with their environment in real time. In such scenarios, the choices themselves as well as their relative costs and benefits are defined by the momentary geometry of the immediate environment and change continuously during ongoing activity. To deal with the demands of embodied activity, animals require an architecture in which the sensorimotor specification of potential actions, their valuation, selection and even execution can all take place in parallel. Here, we review behavioural and neurophysiological data supporting a proposed brain architecture for dealing with such scenarios, which we argue set the evolutionary foundation for the organization of the mammalian brain.

Keywords: action selection; affordances; decision-making; ecological psychology; embodied behaviour.

Figure 1.

Schematic embodied decision-making scenario. Dotted line arrows indicate possible paths for the mouse to move between obstacles depicted by shaded rectangles. Solid curves indicate the distribution of potential directions at three points in time. At point (a), the distribution can be averaged into a single central direction. At point (b) the distribution begins to separate, but averaging is still possible. At point (c), however, the average is no longer a viable direction and a decision must be made between directions to the right or left.

Figure 2.

An experiment on the neural mechanisms of spatial decisions. (a) Task design. In the 1-Target (1T) task, monkeys were presented with a cue whose border style indicated the expected size of reward (see legend at top right) and then given a GO signal to execute the movement. In the 2-Target (2T) task, the monkeys were presented with two possible targets. In 67% of trials (free), they could move to either of these after the GO signal, but in 33% of trials (forced) one disappeared, forcing them to move to the remaining target. (b) Comparison of activity of an example PMd neuron, as a function of reward size, during trials in which the cell's preferred target (PT) was present. In each pair of rasters and peri-event histograms, data are aligned on target onset and GO signal, with a break between them due to delay period variability. Black symbols in the rasters depict target onset, GO signal, and movement onset and offset. If the preferred target was the only one shown (1T task, right), neural activity showed no difference whether its value was low (blue), medium (red) or high (green). However, if two targets were present (2T task), then the same cell showed a strong effect of relative value, both when the value of the preferred target (PT) or the other target (OT) was varied (centre and right). (c) Analysis of the activity of a PMd cell when the PT value was medium and the OT value was varied, done separately depending on the angular distance between targets. Data are aligned on target onset. Note that the modulatory effect of OT value is much stronger when the distance between targets is 120° or 180°, than when they are only 60° apart. (d) Cumulative distributions of the latency with which PMd cells become tuned (green), reflect spatial interactions between targets (blue) and reflect relative value (red). (e) Activity of a PMd cell (aligned on the GO signal) comparing 2T trials in which the monkey was free to choose the higher valued target (free) that was either in the cell's preferred direction (red) or not (purple), and 2T trials in which the monkey was forced to go to the lower valued target (forced low) that was either in the cell's preferred direction (blue) or not (green). Note that activity prior to the GO signal, while both targets were still present, strongly indicates the monkey's preferred plan and activity after the GO signal switches abruptly in forced low trials. Panels (b–d) adapted from Pastor-Bernier & Cisek [88] and panel (e) from Pastor-Bernier et al. [94].

Figure 3.

An experiment on the neural mechanisms of decisions in dynamic situations. (a) Task design. Monkeys were presented with two targets on either side of a central target in which 15 tokens were randomly placed (first row). During each trial, these tokens jumped one-by-one every 200 ms into one or the other target, randomly (second row). The monkeys had to decide which target would receive the majority of the tokens but could make this guess at any time, and after the target was reached (third row), the remaining tokens accelerated and feedback was given when all 15 had jumped (forth row). (b) The time-course of the probability that a target would be correct, during three different trials: in easy trials (blue) probability tended to quickly converge to a target; in ambiguous trials (green) it tended to remain close to 50% for most of the trial; in misleading trials (red), it tended towards the wrong target in the beginning. The grey line shows a schematic ‘decreasing threshold’ which explains behaviour in the task (see text). (c) Decision time distributions in easy (blue), ambiguous (green) and misleading trials (red). Shaded regions indicate error trials and dotted vertical lines indicate means. (d) The average activity of 68 PMd neurons (centre row) and 31 primary motor cortex (M1) neurons (bottom row) reflects the probability profile (top row) in easy, ambiguous and misleading trials when the monkey chooses each cell's preferred target (solid lines) or the opposite target (dotted lines). Neural activity also tends to build up over time in both regions. Data are aligned on the first token jump and, to prevent averaging artefacts, truncated at the estimated movement of decision. (e) Average activity of the same PMd and M1 cells aligned on movement onset. Note the prominent peak of activity in PMd cells tuned to the selected target, which occurs approximately 280 ms before movement onset regardless of the trial type (grey line). Note also that around the same time, there is a sharp suppression of activity in the M1 cells tuned to the unselected target, and a later peak of activity (140 ms before movement onset) in M1 cells tuned to the selected target. Adapted from Thura & Cisek [117].

Figure 4.

An experiment on rapid online correction to unexpected mechanical perturbations. (a) In one variant of the study, subjects made movements from a starting circle (at bottom) to any of three target circles (top) while avoiding obstacles (filled circles). In some trials, a perturbation was applied shortly after movement onset (black arrows). In many trials (blue), subjects moved between the obstacles to the central target, but in other trials (red), especially when the perturbation was medium or strong, they moved around the obstacle and reached the left target. In some trials (red arrow), there was a transient deflection towards the central target. (b) Difference in EMG activity of lateral triceps (an elbow extensor) between the unperturbed condition and the medium-perturbation condition in which subjects went between (blue) or around (red) the obstacle. Data are aligned to perturbation time. R1, R2 and R3 indicate the three components of stretch responses to perturbations, and EV indicates the early voluntary component. (c) Difference between the blue and red curves in (b). Note that there is a significant difference already in the R2 response component, 45–75 ms after perturbation, which is believed to involve the shortest latency transcortical loop. Adapted from Nashed et al. [130].