Unlocking the reinforcement-learning circuits of the orbitofrontal cortex

Abstract

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Figure 1.

Computational framework for understanding the role of OFC circuits in addiction pathology. (A) Adaptive choice behavior is assessed in three-choice, spatial discrimination problems using stochastic reward schedules. The reinforcement probabilities assigned to each noseport are pseudorandomly assigned at the start of each sessions and remain stable for 120 trials (e.g., Acquisition phase). Once rats complete 120 trials, the reinforcement probabilities assigned to each noseport change (e.g., Reversal phase). Rats must adjust their choice behavior following this change to maximize the number of rewards they earn. (B) Rats that have greater difficulty in the reversal phase (e.g., poor reversal performance—orange lines) prior to any drug exposure self-administer more cocaine compared to rats that perform better in the reversal phase (e.g., good reversal performance—gray lines). (C) Cocaine self-administration in 6 h, long-access sessions for 21 days impairs the performance of rats in the reversal phase. (D) Computational analyses of choice behavior collected prior to cocaine self-administration revealed that individual differences in value updating following a positive outcome (i.e., positive value updating) were predictive of the rate of escalation in cocaine self-administration. Viral ablation studies demonstrated that ablation of amygdala (Amy) neurons projecting to the OFC reduces value updating following a positive outcome, suggesting that functional differences in amygdala projections to the OFC may mediate susceptibility to drug taking. (E) A computational analysis of choice behavior before and after cocaine self-administration revealed that value updating following a negative outcome (i.e., negative value updating) was disrupted: rats were more likely to persist with an unrewarded choice following cocaine self-administration. Viral ablation studies demonstrated that ablation of OFC neurons projecting to the nucleus accumbens (NAc) impairs value updating following a negative outcome, suggesting that disruptions in negative value updating observed following cocaine self-administration may be the result of drug-induced disruptions in OFC neurons projecting to the nucleus accumbens. * p < .05. ** p < .01. ** p < .001. This figure is adapted from work described in Groman, Keistler, et al., 2019, and Groman et al., 2020a, .