What is dopamine doing in model-based reinforcement learning?
- PMID: 37082448
- PMCID: PMC7614453
- DOI: 10.1016/j.cobeha.2020.10.010
What is dopamine doing in model-based reinforcement learning?
Abstract
Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine's involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine's involvement in model-based RL.
Figures


References
-
- Schultz W, Dayan P, Montague PR. A Neural Substrate of Prediction and Reward. Science. 1997;275:1593–1599. - PubMed
-
- Sutton RS, Barto AG. Reinforcement learning: An introduction. The MIT press; 1998.
Grants and funding
LinkOut - more resources
Full Text Sources