Direct instrumental conditioning of neural activity using functional magnetic resonance imaging-derived reward feedback
- PMID: 17626211
- PMCID: PMC6672599
- DOI: 10.1523/JNEUROSCI.2118-07.2007
Direct instrumental conditioning of neural activity using functional magnetic resonance imaging-derived reward feedback
Abstract
Successful learning is often contingent on feedback. In instrumental conditioning, an animal or human learns to perform specific responses to obtain reward. Instrumental conditioning is often used by behavioral psychologists to train an animal (or human) to produce a desired behavior. Shaping involves reinforcing those behaviors, which in a stepwise manner are successively closer to the desired behavior until the desired behavior is reached. Here, we aimed to extend this traditional approach to directly shape neural activity instead of overt behavior. To achieve this, we scanned 22 human subjects with functional magnetic resonance imaging and performed image processing in parallel with acquisition. We delineated regions of interest (ROIs) in finger and toe motor/somatosensory regions and used an instrumental shaping procedure to induce a regionally specific increase in activity by providing an explicit monetary reward to reinforce neural activity in the target areas. After training, we found a significant and regionally specific increase in activity in the ROI being rewarded (finger or toe) and a decrease in activity in the nonrewarded region. This demonstrates that instrumental conditioning procedures can be used to directly shape neural activity, even without the production of an overt behavioral response. This procedure offers an important alternative to traditional biofeedback-based approaches and may be useful in the development of future therapies for stroke and other brain disorders.
Figures






Similar articles
-
Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system: an event-related fMRI study.Neuroimage. 2003 Oct;20(2):1086-95. doi: 10.1016/S1053-8119(03)00381-1. Neuroimage. 2003. PMID: 14568478 Clinical Trial.
-
Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain.J Neurophysiol. 2009 Dec;102(6):3384-91. doi: 10.1152/jn.91195.2008. Epub 2009 Sep 30. J Neurophysiol. 2009. PMID: 19793875
-
Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards.Behav Processes. 2010 Jul;84(3):715-20. doi: 10.1016/j.beproc.2010.04.007. Epub 2010 Apr 22. Behav Processes. 2010. PMID: 20417259
-
Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies.Neurosci Biobehav Rev. 2013 Aug;37(7):1297-310. doi: 10.1016/j.neubiorev.2013.03.023. Epub 2013 Apr 6. Neurosci Biobehav Rev. 2013. PMID: 23567522 Review.
-
To work or not to work: Neural representation of cost and benefit of instrumental action.Prog Brain Res. 2016;229:125-157. doi: 10.1016/bs.pbr.2016.06.009. Epub 2016 Aug 31. Prog Brain Res. 2016. PMID: 27926436 Review.
Cited by
-
Comparison of real-time water proton spectroscopy and echo-planar imaging sensitivity to the BOLD effect at 3 T and at 7 T.PLoS One. 2014 Mar 10;9(3):e91620. doi: 10.1371/journal.pone.0091620. eCollection 2014. PLoS One. 2014. PMID: 24614912 Free PMC article.
-
Decoded fMRI neurofeedback can induce bidirectional confidence changes within single participants.Neuroimage. 2017 Apr 1;149:323-337. doi: 10.1016/j.neuroimage.2017.01.069. Epub 2017 Feb 3. Neuroimage. 2017. PMID: 28163140 Free PMC article. Clinical Trial.
-
Putamen volume predicts real-time fMRI neurofeedback learning success across paradigms and neurofeedback target regions.Hum Brain Mapp. 2021 Apr 15;42(6):1879-1887. doi: 10.1002/hbm.25336. Epub 2021 Jan 5. Hum Brain Mapp. 2021. PMID: 33400306 Free PMC article.
-
Differential neural plasticity of individual fingers revealed by fMRI neurofeedback.J Neurophysiol. 2021 May 1;125(5):1720-1734. doi: 10.1152/jn.00509.2020. Epub 2021 Mar 31. J Neurophysiol. 2021. PMID: 33788634 Free PMC article.
-
Covert neurofeedback without awareness shapes cortical network spontaneous connectivity.Proc Natl Acad Sci U S A. 2016 Apr 26;113(17):E2413-20. doi: 10.1073/pnas.1516857113. Epub 2016 Apr 11. Proc Natl Acad Sci U S A. 2016. PMID: 27071084 Free PMC article.
References
-
- Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. - PubMed
-
- Birbaumer N, Kubler A, Ghanayim N, Hinterberger T, Perelmouter J, Kaiser J, Iversen I, Kotchoubey B, Neumann N, Flor H. The thought translation device (TTD) for completely paralyzed patients. IEEE Trans Rehabil Eng. 2000;8:190–193. - PubMed
-
- Chen XY, Wolpaw JR. Operant-conditioning of H-reflex in freely moving rats. J Neurophysiol. 1995;73:411–415. - PubMed
-
- Cox RW, Jesmanowicz A. Real-time 3D image registration for functional MRI. Magn Reson Med. 1999;42:1014–1018. - PubMed
-
- Cox RW, Jesmanowicz A, Hyde JS. Real-time functional magnetic resonance imaging. Magn Reson Med. 1995;33:230–236. - PubMed
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical