Reward improves long-term retention of a motor memory through induction of offline memory gains

Abstract

In humans, training in which good performance is rewarded or bad performance punished results in transient behavioral improvements. The relative effects of reward and punishment on consolidation and long-term retention, critical behavioral stages for successful learning, are not known. Here, we investigated the effects of reward and punishment on these different stages of human motor skill learning. We studied healthy subjects who trained on a motor task under rewarded, punished, or neutral control conditions. Performance was tested before and immediately, 6 hr, 24 hr, and 30 days after training in the absence of reward or punishment. Performance improvements immediately after training were comparable in the three groups. At 6 hr, the rewarded group maintained performance gains, whereas the other two groups experienced significant forgetting. At 24 hr, the reward group showed significant offline (posttraining) improvements, whereas the other two groups did not. At 30 days, the rewarded group retained the gains identified at 24 hr, whereas the other two groups experienced significant forgetting. We conclude that training under rewarded conditions is more effective than training under punished or neutral conditions in eliciting lasting motor learning, an advantage driven by offline memory gains that persist over time.

Figure 1

behavioral task. (a) Tracking isometric pinch force task. Subjects pinched a force transducer between the right thumb and index finger. Squeezing the force transducer resulted in the upward movement of a red cursor on the computer screen, while relaxing caused the cursor to move downward (cursor movements with force shown in shades of red on the monitor). The goal was to maintain the red cursor within the moving blue target by modulating pinch force. Each trial started with the red cursor and the blue target overlapping at the bottom of the screen. (b) Target path. During each trial, the blue target moved in a vertical line for 9 seconds along a consistently repeated trajectory to disappear for 0.5 s at the end of the trial. The y-axis displays the vertical distance (cm) from the lower edge of the blue box and the bottom of the screen. The x-axis shows the elapsed time (s). Intertrial intervals varied randomly between 1 and 2 seconds. (c) Mean error calculation. Error was defined as the vertical distance between the edges of the blue target and the red cursor at each sampled time point, as shown. Each single-trial error was calculated by averaging the errors along all sampled time points in the trajectory (sampled every 20 ms for 9 s). Mean error was calculated as the mean across all 10 trials within each test block.

Figure 2

Experimental design. Subjects participated in 3 different sessions (days 1, 2 and 30) separated into 3 training groups who practiced the task over 4 blocks (20 trials each, black rectangle) under the influence of monetary reward (green, n = 13), monetary punishment (red, n = 12) or neutral conditions (blue, n = 12) in a factorial design. During training blocks, monetary reward, monetary punishment, or neutral visual information was presented for 1 s at the end of each trial depending on the group. Test blocks were evaluated in all subjects in the absence of reward/punishment/neutral information before training (baseline), and immediately, 6, 24 hs and 30 days after training.

Figure 3

(a) Effect of reward and punishment on motor skill. Mean errors in the rewarded, punished and neutral groups as a function of time. A repeated measures mixed-model ANOVA with factors GROUP (rewarded/punished/neutral) and TIME (baseline/immediate/6 hours/24 hours/30 days) on mean error showed a significant effect of TIME (F(4, 140) = 403.1, p < 0.001) and a trend for GROUP (F(2, 35) = 2.72, p = 0.08). Most importantly, there was a significant GROUP x TIME interaction (F(8, 140) = 4.41, p < 0.001), indicating a different time course of performance changes across groups. At baseline and immediately after training, all groups had similar mean errors. Of note are the comparable mean errors immediately after training and the significantly smaller mean error at 30 days post-training in the rewarded relative to the neutral and punished groups, a difference that started to develop as early as 6 hours post-training. (b) Effect of reward and punishment on motor skill retention after training. Changes in mean error (delta) between the immediate and the other (6 hs, 24 hs or 30 days) post-training time points. Values above 0 indicate decrease in mean error (improved performance), while those below 0 indicate increase in mean error (worsened performance). Of note is the significantly greater delta at 30 days post-training in the rewarded relative to the neutral and punished groups, a difference that started to develop as early as 6 hours post training. Data shown as delta ± SEM. Asterisks indicate corrected p < 0.05.

Figure 3

(a) Effect of reward and punishment on motor skill. Mean errors in the rewarded, punished and neutral groups as a function of time. A repeated measures mixed-model ANOVA with factors GROUP (rewarded/punished/neutral) and TIME (baseline/immediate/6 hours/24 hours/30 days) on mean error showed a significant effect of TIME (F(4, 140) = 403.1, p < 0.001) and a trend for GROUP (F(2, 35) = 2.72, p = 0.08). Most importantly, there was a significant GROUP x TIME interaction (F(8, 140) = 4.41, p < 0.001), indicating a different time course of performance changes across groups. At baseline and immediately after training, all groups had similar mean errors. Of note are the comparable mean errors immediately after training and the significantly smaller mean error at 30 days post-training in the rewarded relative to the neutral and punished groups, a difference that started to develop as early as 6 hours post-training. (b) Effect of reward and punishment on motor skill retention after training. Changes in mean error (delta) between the immediate and the other (6 hs, 24 hs or 30 days) post-training time points. Values above 0 indicate decrease in mean error (improved performance), while those below 0 indicate increase in mean error (worsened performance). Of note is the significantly greater delta at 30 days post-training in the rewarded relative to the neutral and punished groups, a difference that started to develop as early as 6 hours post training. Data shown as delta ± SEM. Asterisks indicate corrected p < 0.05.

Figure 4

Time course of memory changes. Online gains were comparable in the three groups. While the reward group (green) experienced substantial offline memory gains, the other two groups did not. By 30 days memory in the rewarded group stabilized offline gains while in the other two groups deteriorated. The neural structures mediating different stages of rewarded learning remain to be determined firmly but may include the cerebellum, neocortex and striatum in the intersection of networks that provide substrate for learning of this task [34] and processing of reward information [32, 35, 36]. It remains to be determined to which extent these observations apply to motor adaptation paradigms [37, 38]. It is conceivable that the cerebellum could contribute to error-based learning while the striatum and neocortex may become engaged in later stages [39, 40] and long-term retention [41] under rewarded conditions.