Vigor of reaching movements: reward discounts the cost of effort

Abstract

Making a movement may be thought of as an economic decision in which one spends effort to acquire reward. Time discounts reward, which predicts that the magnitude of reward should affect movement vigor: we should move faster, spending greater effort, when there is greater reward at stake. Indeed, saccade peak velocities are greater and reaction-times shorter when a target is paired with reward. In this study, we focused on human reaching and asked whether movement kinematics were affected by expectation of reward. Participants made out-and-back reaching movements to one of four quadrants of a 14-cm circle. During various periods of the experiment only one of the four quadrants was paired with reward, and the transition from reward to nonreward status of a quadrant occurred randomly. Our experiment design minimized dependence of reward on accuracy, granting the subjects wide latitude in self-selecting their movement speed, amplitude, and variability. When a quadrant was paired with reward, reaching movements had a shorter reaction time, higher peak velocity, and greater amplitude. Despite this greater vigor, movements toward the rewarded quadrant suffered from less variability: both reaction times and reach kinematics were less variable when there was expectation of reward. Importantly, the effect of reward on vigor was specific to the movement component that preceded the time of reward (outward reach), not the movement component that followed it (return reach). Our results suggest that expectation of reward not only increases vigor of human reaching but also decreases its variability. NEW & NOTEWORTHY Movements may be thought of as an economic transaction where the vigor of the movement represents the effort that the brain is willing to expend to acquire a rewarding state. We show that in reaching, reward discounts the cost of effort, producing movements with shorter reaction time, higher velocity, greater amplitude, and reduced reaction-time variability. These results complement earlier observations in saccades, suggesting a common principle of economics across modalities of motor control.

Keywords: effort; reaching movements; reward; variability; vigor.

Fig. 1.

Experimental design. A: setup. Participants sat in a chair while grasping the handle of a robotic arm that controlled a cursor on a monitor located at eye level. A shoulder harness was used to prevent movement of the trunk during the reaching task. B: movement metrics. For each trial, participants completed out-and-back reaches to 1 of 4 alternating targets located 14 cm from the home circle. Reaction time, peak outward velocity, crossing point, maximum excursion, duration, and peak return velocity were recorded for each movement. C: experimental protocol. The experiment consisted of a baseline period of 40 trials with no visual feedback or reward, followed by 4 blocks of 100 trials. Each block had one target paired with a reward (RWD; indicated by quadrant with shaded gray region). The reward consisted of an exploding target, auditory stimulus, and 4 points. The order of rewarded blocks was randomized for each participant. D: position data to each target for a single participant (S3).

Fig. 2.

Movement characteristics. A: probability distribution of reaction time was estimated for each subject in each condition using a nonparametric approach (bin size = 5 ms). The change in reaction time is a within-subject measure. Mode of the reaction time appeared to shift earlier, and the variance appeared to decrease. B: radial position trajectory and the change in radial position as a function of time. The hand appeared to reach farther in the rewarded condition. C: radial velocity and the change in radial velocity as a function of time. The hand appeared to reach faster in the rewarded condition. Because of the range of movement durations selected across participants, group averages are displayed up to the point of the shortest individual curve. Shaded regions are ±SE. D: delta plot of reaction time across 20% quantiles. For each subject, reaction times in each condition were rank ordered and sorted into 20% quantiles. Values along the x-axis represent mean reaction time for no reward (NRWD) at each quantile. Values along the y-axis represent the change in the mean from reward (RWD) to NRWD condition. Negative values indicate that reward decreased reaction times, and the negative slope suggests that reward reduced the variance of reaction times. Error bars are ±SE.

Fig. 3.

Within-subject measures on the effects of reward. Reward-dependent changes in mean of the reaction times (A), variance of the reaction times (B), peak velocity of the outward movement (C), maximum excursion (D), and duration of the outward movement (E) are shown. In bar plots, gray bars represent within-subject change and black bars represent group means ± SE. *P < 0.05; **P < 0.01; ***P < 0.001. Participants were ranked according to their relative change in peak velocity (C). Differences represent reward minus nonreward. In scatter plots, black dots represent individual participants and black lines represent group means ± SE.

Fig. 4.

Effect of block number on reaching movements. Effects of block on peak outward velocity (A), reaction time (B), maximum excursion (C), duration (D), crossing-point distance (E), reaction-time variance (F), and crossing-point variance (G) are shown. Black and gray lines represent rewarded and nonrewarded trials, respectively. Bars are means ± SE. Graphs have been slightly offset horizontally to improve contrast. Results from post hoc comparisons regarding the main effect of block are based on averages combining both rewarded and nonrewarded movements. *P < 0.05; **P < 0.01. Subset graphs represent within-subject difference (Δ) calculated as reward minus nonreward at each block.

Fig. 5.

Trial-to-trial effect of reward. Changes in peak outward velocity (A), reaction time (B), maximum excursion (C), duration (D), crossing-point distance (E), reaction-time variance (F), and crossing-point variance (G) as a result of reward on subsequent and preceding nonrewarded targets are shown. *P < 0.05; **P < 0.01; ***P < 0.001 compared with reward; n.s., not significant. All reported values are relative to the rewarded trial. Bars are means ± SE.