Hemispheric Asymmetries in Striatal Reward Responses Relate to Approach-Avoidance Learning and Encoding of Positive-Negative Prediction Errors in Dopaminergic Midbrain Regions

Abstract

Some individuals are better at learning about rewarding situations, whereas others are inclined to avoid punishments (i.e., enhanced approach or avoidance learning, respectively). In reinforcement learning, action values are increased when outcomes are better than predicted (positive prediction errors [PEs]) and decreased for worse than predicted outcomes (negative PEs). Because actions with high and low values are approached and avoided, respectively, individual differences in the neural encoding of PEs may influence the balance between approach-avoidance learning. Recent correlational approaches also indicate that biases in approach-avoidance learning involve hemispheric asymmetries in dopamine function. However, the computational and neural mechanisms underpinning such learning biases remain unknown. Here we assessed hemispheric reward asymmetry in striatal activity in 34 human participants who performed a task involving rewards and punishments. We show that the relative difference in reward response between hemispheres relates to individual biases in approach-avoidance learning. Moreover, using a computational modeling approach, we demonstrate that better encoding of positive (vs negative) PEs in dopaminergic midbrain regions is associated with better approach (vs avoidance) learning, specifically in participants with larger reward responses in the left (vs right) ventral striatum. Thus, individual dispositions or traits may be determined by neural processes acting to constrain learning about specific aspects of the world.

Keywords: approach; asymmetry; avoidance; dopamine; reinforcement; reward.

Figure 1.

A, One training trial in the probabilistic selection task. After fixation, two symbols were presented and participants selected one symbol within 1 s. After 1 s, positive or negative feedback was presented based on the reward probability associated with the selected symbol. RT, Response time. B, Reward probabilities associated with each pair and symbol. The symbols associated with each reward probability were randomized between participants.

Figure 2.

A, Performance as a function of training (mean ± SEM). Participants improved performance as training progressed, as indicated by increased performance at the end of training. Performance increased most rapidly for AB pairs where the difference in reward probability between the symbols was largest. B, Performance, as derived from the reinforcement learning model, as a function of training. Overall, the predicted performance (solid lines) provides a good fit to the observed performance (dots). C, Selection rate during the test phase with novel pairs (mean ± SEM). Symbols with higher and lower reward probabilities during the training were more likely to be selected and avoided in the novel pairs. D, Selection rate, as derived from the reinforcement learning model, during the test phase with novel pairs (mean ± SEM). Overall, the predicted performance (white bars) provides a good fit to the observed performance (black dots).

Figure 3.

A, Neural response to positive compared with negative feedback was significantly larger in both the left and right NAcc. Red circles represent the left and right NAcc ROIs. B, Average beta value (mean ± SEM) for positive relative negative feedback extracted from 3-mm-radius spheres centered on the coordinates of the peak voxel within the left and right NAcc ROIs. **p < 0.001. ^•p < 0.1. C, Distribution of individual differences in hemispheric reward asymmetry. Fourteen participants displayed larger reward responsiveness in the left NAcc, whereas 20 participants displayed larger reward responsiveness in the right NAcc. D, Learning bias (approach vs avoidance learning) as a function of hemispheric reward asymmetry. Participants with a relatively larger relative reward response in the left (resp. right) NAcc displayed relatively more approach (resp. avoidance) learning. HS, Hemisphere.

Figure 4.

A, Neural activity within the SN/VTA ROI tracked net PEs. Red circle represents the SN/VTA ROI. B, The neural correlate of net PEs overlaid on an average proton density structural image. The substantia nigra can be identified as a white strip surrounding the ventral tegmental area. C, Neural activity in the dopaminergic midbrain (MNI: x = 6, y = 16, z = −5) correlating with negative and positive PEs (mean ± SEM). **p < 0.01. ^•p < 0.1. ns., Not significant. D, Relative encoding of positive versus negative PEs as a function of hemispheric reward asymmetry. Participants with a relatively larger reward response in the left (vs the right) NAcc displayed relatively better encoding of positive (vs negative) PEs. HS, Hemisphere.