Direct instrumental conditioning of neural activity using functional magnetic resonance imaging-derived reward feedback

Abstract

Successful learning is often contingent on feedback. In instrumental conditioning, an animal or human learns to perform specific responses to obtain reward. Instrumental conditioning is often used by behavioral psychologists to train an animal (or human) to produce a desired behavior. Shaping involves reinforcing those behaviors, which in a stepwise manner are successively closer to the desired behavior until the desired behavior is reached. Here, we aimed to extend this traditional approach to directly shape neural activity instead of overt behavior. To achieve this, we scanned 22 human subjects with functional magnetic resonance imaging and performed image processing in parallel with acquisition. We delineated regions of interest (ROIs) in finger and toe motor/somatosensory regions and used an instrumental shaping procedure to induce a regionally specific increase in activity by providing an explicit monetary reward to reinforce neural activity in the target areas. After training, we found a significant and regionally specific increase in activity in the ROI being rewarded (finger or toe) and a decrease in activity in the nonrewarded region. This demonstrates that instrumental conditioning procedures can be used to directly shape neural activity, even without the production of an overt behavioral response. This procedure offers an important alternative to traditional biofeedback-based approaches and may be useful in the development of future therapies for stroke and other brain disorders.

Figure 1.

a, Example time course for a conditioning trial. Subjects were presented with a resting cue for a variable interval between 10 and 20 s, followed by a cue to activate a specific brain region for 15 s. A percentage signal change value from resting to active was computed on-line and compared with the current threshold. If the threshold was exceeded, subjects were shown a picture of a dollar bill, indicating that they had won one dollar, otherwise a scrambled picture of a dollar was shown, for 2 s. b, Diagram showing typical fMRI slice coverage, overlaid on a sagittal slice from a single subject's anatomical scan. We imaged 16 3 mm slices, straight across the top of cortex.

Figure 2.

Sample movement recordings from a single subject during experiment 1. a, EMG during real hand movement. b, EMG during imagined hand movement. c, Variable resistor recording during real foot movement. d, Variable resistor recording during imagined foot movement.

Figure 3.

fMRI results from experiment 1. a, Mean percentage signal change data averaged over subjects within runs in each ROI during trials in which the foot ROI was rewarded. b, Difference in mean percentage signal change data, averaged over subjects within runs, between foot and hand ROIs during foot rewarded trials. c, Results of random-effects analysis in SPM from experiment 1; t test on contrast increasing during foot rewarded trials and decreasing during hand rewarded trials, thresholded at p < 0.01; cross-hairs indicate mean of subjects' ROI centers for the foot ROI [−6, −25, 69]. d, Averaged responses in each ROI during trials in which the hand ROI was rewarded. e, Difference between hand and foot ROIs during hand rewarded trials. f, Results of random-effects analysis in SPM from experiment 1; t test on contrast increasing during hand rewarded trials and decreasing during foot rewarded trials, thresholded at p < 0.001; cross-hairs indicate mean of subjects' ROI centers for the hand ROI [−35, −26, 65]. Error bars indicate SE. WB, Whole brain background.

Figure 4.

Subject-averaged parameter estimates across sessions from experiment 1. Hand and foot rewarded trials are plotted separately. Error bars indicate SEs. Regional coordinates are as in Table 2. a, Regions identified as significant during trials when subjects were rewarded for activating the foot region. b, Regions identified as significant during trials when subjects were rewarded for activating the hand region. MFG, Middle frontal gyrus; SMG, supramarginal gyrus; SPG, superior parietal gyrus.

Figure 5.

Percentage signal change plots across sessions for feedback and control groups from experiment 2. a, Difference in percentage signal change between foot and hand ROIs when foot responses were rewarded. b, Difference in percentage change between hand and foot ROIs when hand responses were rewarded. Error bars indicate SE.

Figure 6.

Random-effects and ROI analyses from experiment 2. a, Results of a contrast of foot-cue versus hand-cue conditions across all four sessions. Cross-hairs are centered on mean of subjects' ROI centers for the foot ROI [−6, −30, 69]. Results are shown at p < 0.01 for visualization but survive correction for small volume at p < 0.05. b, Results of a contrast of hand-cue rewarded versus foot-cue rewarded trials across all four sessions from the feedback group. Cross-hairs are centered on mean of subjects' ROI centers for the hand ROI [−39, −27, 57]. Results are shown at p < 0.01 for visualization but survive correction for small volume at p < 0.05.