Attentional Orienting by Non-informative Cue Is Shaped via Reinforcement Learning

Abstract

It has been demonstrated that a reward-associated stimulus feature captures attention involuntarily. The present study tested whether spatial attentional orienting is biased via reinforcement learning. Participants were to identify a target stimulus presented in one of two placeholders, preceded by a non-informative arrow cue at the center of the display. Importantly, reward was available when the target occurred at a location cued by a reward cue, defined as a specific color (experiments 1 and 3) or a color-direction combination (experiment 2). The attentional bias of the reward cue was significantly increased as trials progressed, resulting in a greater cue-validity effect for the reward cue than the no-reward cue. This attentional bias was still evident even when controlling for the possibility that the incentive salience of the reward cue color modulates the cue-validity effect (experiment 2) or when the reward was withdrawn after reinforcement learning (experiment 3). However, it disappeared when the reward was provided regardless of cue validity (experiment 4), implying that the reinforcement contingency between reward and attentional orienting is a critical determinant of reinforcement learning-based spatial attentional modulation. Our findings highlight that a spatial attentional bias is shaped by value via reinforcement learning.

Keywords: attentional bias; attentional orienting; reinforcement learning; spatial attention; value-driven attention.

FIGURE 1

An example of a trial sequence in experiment 1.

FIGURE 2

(A) Mean reaction time (in milliseconds) as a function of cue type and validity in experiment 1. (B) Mean reaction time (in milliseconds) as a function of block, cue type, and validity in experiment 1. Error bars ± within-subject standard error of the mean (Cousineau, 2005).

FIGURE 3

An example of a trial sequence in experiment 2.

FIGURE 4

Mean reaction times (in milliseconds) as a function of block, cue type, and validity in experiment 2. Error bars ± 1 within-subject standard error of the mean (Cousineau, 2005).

FIGURE 5

Examples of a trial sequence of the learning phase (A) and the test phase (B) in experiment 3.

FIGURE 6

Mean reaction times (in milliseconds) as a function of block, cue type, and validity in the learning phase (left) and the test phase (right) of experiment 3. Error bars ± 1 within-subject standard error of the mean (Cousineau, 2005).

FIGURE 7

Mean reaction times (in milliseconds) as a function of block, cue type, and validity in the learning phase (left) and the test phase (right) of experiment 4. Error bars ± 1 within-subject standard error of the mean (Cousineau, 2005).

FIGURE 8

(A) An example of the competitive relationship between the Pavlovian and instrumental response-based orienting. (B) An example of the confounding relationship between them. (C) An example of dissociated attentional orienting based on the Pavlovian and instrumental response.