Direct instrumental conditioning of neural activity using functional magnetic resonance imaging-derived reward feedback
- PMID: 17626211
- PMCID: PMC6672599
- DOI: 10.1523/JNEUROSCI.2118-07.2007
Direct instrumental conditioning of neural activity using functional magnetic resonance imaging-derived reward feedback
Abstract
Successful learning is often contingent on feedback. In instrumental conditioning, an animal or human learns to perform specific responses to obtain reward. Instrumental conditioning is often used by behavioral psychologists to train an animal (or human) to produce a desired behavior. Shaping involves reinforcing those behaviors, which in a stepwise manner are successively closer to the desired behavior until the desired behavior is reached. Here, we aimed to extend this traditional approach to directly shape neural activity instead of overt behavior. To achieve this, we scanned 22 human subjects with functional magnetic resonance imaging and performed image processing in parallel with acquisition. We delineated regions of interest (ROIs) in finger and toe motor/somatosensory regions and used an instrumental shaping procedure to induce a regionally specific increase in activity by providing an explicit monetary reward to reinforce neural activity in the target areas. After training, we found a significant and regionally specific increase in activity in the ROI being rewarded (finger or toe) and a decrease in activity in the nonrewarded region. This demonstrates that instrumental conditioning procedures can be used to directly shape neural activity, even without the production of an overt behavioral response. This procedure offers an important alternative to traditional biofeedback-based approaches and may be useful in the development of future therapies for stroke and other brain disorders.
Figures






Similar articles
-
Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system: an event-related fMRI study.Neuroimage. 2003 Oct;20(2):1086-95. doi: 10.1016/S1053-8119(03)00381-1. Neuroimage. 2003. PMID: 14568478 Clinical Trial.
-
Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain.J Neurophysiol. 2009 Dec;102(6):3384-91. doi: 10.1152/jn.91195.2008. Epub 2009 Sep 30. J Neurophysiol. 2009. PMID: 19793875
-
Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards.Behav Processes. 2010 Jul;84(3):715-20. doi: 10.1016/j.beproc.2010.04.007. Epub 2010 Apr 22. Behav Processes. 2010. PMID: 20417259
-
Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies.Neurosci Biobehav Rev. 2013 Aug;37(7):1297-310. doi: 10.1016/j.neubiorev.2013.03.023. Epub 2013 Apr 6. Neurosci Biobehav Rev. 2013. PMID: 23567522 Review.
-
To work or not to work: Neural representation of cost and benefit of instrumental action.Prog Brain Res. 2016;229:125-157. doi: 10.1016/bs.pbr.2016.06.009. Epub 2016 Aug 31. Prog Brain Res. 2016. PMID: 27926436 Review.
Cited by
-
Connectivity-based neurofeedback: dynamic causal modeling for real-time fMRI.Neuroimage. 2013 Nov 1;81:422-430. doi: 10.1016/j.neuroimage.2013.05.010. Epub 2013 May 11. Neuroimage. 2013. PMID: 23668967 Free PMC article.
-
Individual- and Connectivity-Based Real-Time fMRI Neurofeedback to Modulate Emotion-Related Brain Responses in Patients with Depression: A Pilot Study.Brain Sci. 2022 Dec 14;12(12):1714. doi: 10.3390/brainsci12121714. Brain Sci. 2022. PMID: 36552173 Free PMC article.
-
Optimizing real time fMRI neurofeedback for therapeutic discovery and development.Neuroimage Clin. 2014 Jul 10;5:245-55. doi: 10.1016/j.nicl.2014.07.002. eCollection 2014. Neuroimage Clin. 2014. PMID: 25161891 Free PMC article. Review.
-
A Guide to Literature Informed Decisions in the Design of Real Time fMRI Neurofeedback Studies: A Systematic Review.Front Hum Neurosci. 2020 Feb 25;14:60. doi: 10.3389/fnhum.2020.00060. eCollection 2020. Front Hum Neurosci. 2020. PMID: 32161529 Free PMC article.
-
Can we predict real-time fMRI neurofeedback learning success from pretraining brain activity?Hum Brain Mapp. 2020 Oct 1;41(14):3839-3854. doi: 10.1002/hbm.25089. Epub 2020 Jul 30. Hum Brain Mapp. 2020. PMID: 32729652 Free PMC article.
References
-
- Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. - PubMed
-
- Birbaumer N, Kubler A, Ghanayim N, Hinterberger T, Perelmouter J, Kaiser J, Iversen I, Kotchoubey B, Neumann N, Flor H. The thought translation device (TTD) for completely paralyzed patients. IEEE Trans Rehabil Eng. 2000;8:190–193. - PubMed
-
- Chen XY, Wolpaw JR. Operant-conditioning of H-reflex in freely moving rats. J Neurophysiol. 1995;73:411–415. - PubMed
-
- Cox RW, Jesmanowicz A. Real-time 3D image registration for functional MRI. Magn Reson Med. 1999;42:1014–1018. - PubMed
-
- Cox RW, Jesmanowicz A, Hyde JS. Real-time functional magnetic resonance imaging. Magn Reson Med. 1995;33:230–236. - PubMed
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical