Reward Prediction Error Modulates Saccade Vigor

Abstract

Movement vigor, defined as the reciprocal of the latency from availability of reward to its acquisition, changes with reward magnitude: movements exhibit shorter reaction time and increased velocity when they are directed toward more rewarding stimuli. This invigoration may be due to release of dopamine before movement onset, which has been shown to be modulated by events that signal reward prediction error (RPE). Here, we generated an RPE event in the milliseconds before movement onset and tested whether there was a relationship between RPE and vigor. Human subjects (both sexes) made saccades toward an image. During execution of the primary saccade, we probabilistically changed the position and content of that image, encouraging a secondary saccade. On some trials, the content of the secondary image was more valuable than the first image, resulting in a positive RPE (+RPE) event that preceded the secondary saccade. On other trials, this content was less valuable (-RPE event). We found that reaction time of the secondary saccade was affected in an orderly fashion by the magnitude and direction of the preceding RPE event: the most vigorous saccades followed the largest +RPE, whereas the least vigorous saccades followed the largest -RPE. Presence of the secondary saccade indicated that the primary saccade had experienced a movement error, inducing trial-to-trial adaptation. However, this learning from movement error was not modulated by the RPE event. The data suggest that RPE events, which are thought to transiently alter the release of dopamine, modulate the vigor of the ensuing movement.SIGNIFICANCE STATEMENT Does dopamine release in response to a stimulus serve to invigorate the ensuing movement? To test this hypothesis, we relied on the fact that reward prediction error (RPE) is a strong modulator of dopamine. Our innovation was a task in which an RPE event occurred precisely before onset of a stimulus-driven movement. We probabilistically produced a combination of large or small, negative or positive RPE events and observed that saccade vigor carried a robust signature of the preceding RPE event: high vigor saccades followed +RPE events, whereas low vigor saccades followed -RPE events. This suggests that in humans, vigor is partly controlled through release of dopamine in the moments before onset of a movement.

Keywords: dopamine; latency; motor control; reward prediction error; saccade; vigor.

Figure 1.

Experiment design and data from a representative subject. A, Each trial began with a fixation dot near the center. After a random fixation interval, we presented a primary image at 9° to the right along the horizontal axis. As the primary saccade took place, we erased the primary image and replaced it with a secondary image. In +RPE trials, a noise primary image was replaced with a face secondary image. A face–face trial served as control for the +RPE trial. In −RPE trials, a face primary image was replaced with a noise secondary image. A noise–noise trial served as control for the −RPE trial. B, Saccade position and velocity traces for a representative subject. Position traces are for individual saccades. Velocity traces are within-subject average of all saccades. Primary saccade exhibited a shorter reaction time and a higher velocity in response to a face image. Data for the secondary saccade are plotted with respect to termination of the primary saccade in the same trial. The secondary saccades exhibited the shortest reaction times in +RPE trials and the longest reaction times in −RPE trials. Error bars indicate SEM.

Figure 2.

Primary saccades exhibited shorter reaction time and higher velocity in response to face images. A, Distribution of mean reaction times and within-subject change in the mean reaction time (face minus noise). Each dot is a single subject. B, Saccade velocity traces and within-subject change in velocity (face minus noise). The trace for noise is largely hidden behind the trace for face. C, Total time to target (reaction time plus movement duration) and within-subject change. Error bars and shaded traces indicate SEM.

Figure 3.

Secondary saccades were influenced by the preceding RPE event. A, Saccade velocity and within-subject change in velocity. B, Distribution of mean reaction times across subjects. Reaction times are measured as the latency with respect to offset of the preceding primary saccade. C, Mean reaction times across subjects and the within-subject change in reaction times. The bars are the +RPE condition with respect to control FF and the −RPE condition with respect to control NN. Dots are individual subjects. D, Peak velocity and within-subject change in peak velocity. E, Total time to target measured from completion of the primary saccade to conclusion of the secondary saccade; that is, the reaction time plus movement duration of the secondary saccade. The error bars and shaded traces indicate SEM. The dots show within-subject change with respect to control.

Figure 4.

Learning from movement errors and the effect of RPE on learning. A, Change in the velocity of the primary saccade from the trial in which the movement error was experienced to the next trial (the primary image type in both saccades were the same). Saccades are grouped based on the movement error experienced at the conclusion of the first primary saccade. Movement error is defined as the position of the secondary image with respect to the primary image. That is, H+ errors imply that the secondary image was further to the right along the horizontal axis than the primary image. The motor error along each axis produced learning along that axis. B, Trial-to-trial changes for the horizontal motor-errors-only grouped based on the RPE event that followed the first primary saccade. Data for H− were flipped and averaged with H+ trials. Within-subject change in learning was marginally larger after a +RPE event (compared with a −RPE event), but this effect was not significant.