Modeling uncertainty-seeking behavior mediated by cholinergic influence on dopamine

Abstract

Recent findings suggest that acetylcholine mediates uncertainty-seeking behaviors through its projection to dopamine neurons - another neuromodulatory system known for its major role in reinforcement learning and decision-making. In this paper, we propose a leaky-integrate-and-fire model of this mechanism. It implements a softmax-like selection with an uncertainty bonus by a cholinergic drive to dopaminergic neurons, which in turn influence synaptic currents of downstream neurons. The model is able to reproduce experimental data in two decision-making tasks. It also predicts that: (i) in the absence of cholinergic input, dopaminergic activity would not correlate with uncertainty, and that (ii) the adaptive advantage brought by the implemented uncertainty-seeking mechanism is most useful when sources of reward are not highly uncertain. Moreover, this modeling work allows us to propose novel experiments which might shed new light on the role of acetylcholine in both random and directed exploration. Overall, this study contributes to a more comprehensive understanding of the role of the cholinergic system and, in particular, its involvement in decision-making.

Keywords: Acetylcholine; Decision-making; Dopamine; Uncertainty.