Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

Abstract

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Keywords: Behavioral activation; Cost-benefit computation; Delay-discounting; Effort-based decision making; Motivation; Risk-discounting.

Figure 1. Hypothetical Model of Factors Influencing Motivational Cost-Benefit Decision Making Processes

Shows a hypothetical model of how motivation is influenced by physiological state, environment, and past history to modulate an underlying cost-benefit decision making computation which gives direction and vigor to goal-directed behavior

Figure 2. Effects of Methamphetamine in a PR and PHD Task

(A–B). Shows a schematic representation of the Progressive Ratio Task (A) and the Progressive Hold Down Task (B). The yellow bars represent lever presses in (A), and lever holds in (B). The red arrows signify rewards. (C). IP administration of 1.0mg/kg of methamphetamine leads to significant increases in lever pressing and breakpoint in a progressive ratio schedule of reinforcement. (D). IP administration of 1.0mg/kg of methamphetamine leads to significant increase in the number of hold attempts in a PHD task, but does not lead to a significant increase in the highest duration requirement completed (BP). (E). Data from a single subject treated with methamphetamine or vehicle in the PHD task demonstrates the increased number of responses which occur while on the drug, but the responses are inefficient and of shorted durations that required by the schedule. Data in (C – E) from Bailey et al., 2015.

Figure 3. Brain Regions Which Impact Performance on a Progressive Ratio and Effort Based Choice

A. Shows brain regions which have been studied to examine their involvement in PR performance through lesion, inactivation, or localized drug infusion studies which have been shown to modulate PR behavior (Orange), have no effect on PR behavior (Grey), or have yet to be examined (White). Areas which have been shown to modulate PR behavior include: the VTA, the Ventral hippocampus, the SN pars reticulata and SN par compata, the NAcc Core, the OFC, and the PR/IL cortex. Areas which have been studies, but damage or inactivation had no impact on PR performance include: ACC, and NAcc Shell. All other areas have not been studied with a PR task: VP, DS. IL, Hipp B. Shows brain regions which have been studied to examine their involvement in effort based choice performance through lesion, inactivation, or localized drug infusion studies which have been shown to modulate effort choice behavior (Green), have no effect (Grey), or have yet to be examined (White). Areas which have been shown to modulate effort based choice include: the VTA (Reference), NAcc Core, NAcc Shell, VP, ACC. Areas which have been studies, but damage or inactivation had no impact include: OFC, PL, IL. Areas which have yet to be studied include: DS, STN, SN, Hipp, vSyb

Figure 4. Brain Regions involved in Delay and Risky Choice

A. Shows brain regions which have been studied to examine their involvement in delay based choice through lesions, inactivation, or localized drug infusions which have been shown to modulate delay based choice (Blue), have no effect on delay based choice (Grey), or have not yet been examine (White). Areas which influence delay based choice include: the PL, IL, OFC, NAcc Core, BLA, STN, and VTA. Areas which do not influence delay based choice include: the ACC, and NAcc Shell. Areas which have not been studied include: VP, DS, Hipp, SN, vSub. B. Shows brain regions which have been studied to examine their involvement in risk based choice through lesions, inactivation, or localized drug infusions which have been shown to modulate delay based choice (Red), have no effect on delay based choice (Grey), or have not yet been examine (White). Areas which influence delay based choice include: the PL, NAcc Shell, BLA, and VTA. Areas which do not influence delay based choice include: the ACC, OFC, and NAcc Core. Areas which have not been studied include: IL, VP, DS, Hipp, STN, SN, vSub.

Figure 5. Summary Brain Regions and Types of Costs

Shows an overall summary of the brain regions which have been shown to modulate different aspects of motivated behavior and overcoming response costs through lesion studies, chemical inactivation, and pharmacological manipulations. (A) Shows brain regions which have been shown to be modulate vigor in a PR (orange) have no effect on PR (grey), and have not been studied (white). (B) Shows brain regions which have been shown to be involved in effort based choice behavior (green), have no effect on effort based choice (grey), and have not be studied (white). (C) Shows brain regions which have been shown to be involved in delay based choice behavior (blue), have no effect on delay based choice (grey), and have not been studied (white). (D) Shows brain regions which have been shown to be involved in risk based choice behavior (red), have no effect on risk based choice (grey), and have not been studied (white). (E) Provides an overall summary of the different types of behavioral tasks each of the different brain regions has been shown to be involved in.