Interactions of timing and prediction error learning

Abstract

Timing and prediction error learning have historically been treated as independent processes, but growing evidence has indicated that they are not orthogonal. Timing emerges at the earliest time point when conditioned responses are observed, and temporal variables modulate prediction error learning in both simple conditioning and cue competition paradigms. In addition, prediction errors, through changes in reward magnitude or value alter timing of behavior. Thus, there appears to be a bi-directional interaction between timing and prediction error learning. Modern theories have attempted to integrate the two processes with mixed success. A neurocomputational approach to theory development is espoused, which draws on neurobiological evidence to guide and constrain computational model development. Heuristics for future model development are presented with the goal of sparking new approaches to theory development in the timing and prediction error fields.

Keywords: Computational modeling; Motivation; Prediction error learning; Timing.

Figure 1

Mean response rate (in responses/min) of head entry responses as a function of mean interval duration. Filled triangles depict data from delay conditioning studies with a noise CS and open squares are data from light CSs. Plus signs are response rates during the inter-trial interval. A single power function is fit through the data, and the equation and goodness of fit (R²) are provided. The data are combined and adapted from Jennings, Bonardi and Kirkpatrick (2007) and Kirkpatrick (2002).

Figure 2

Top: Mean (± SEM) start times during original baseline (BASE 1), magnitude shift (MAG) and return to baseline (BASE 2) phases of training in Groups 4-1-4 and 1-4-1. The data are adapted from Galtress and Kirkpatrick (2009). Bottom: Mean (+ SEM) start times during baseline (BASE) and test (TEST) phases in groups that were pre-fed in original baseline and tested under deprivation (Pre-Fed) or were trained in the baseline phase under deprivation and tested under satiety (Non-Fed). The data are adapted from Galtress, Marshall, and Kirkpatrick (2012).

Figure 3

A diagram showing prediction error learning (dashed lines), timing (solid lines), and overlapping/shared (double-dashed lines) neural substrates. VTA = ventral tegmental area; NA = nucleus accumbens; BLA = basolateral amygdala; SNc = substantia nigra pars compacta; DS = dorsal striatum; C = Caudate; Pu = Putamen; TH = thalamus; GPe = external segment of globus pallidus; STN = subthalamic nucleus; SNr = substantia nigra pars reticula; GPi = internal segment of globus pallidus. Multiple cortical areas are represented as “Cortex” and include the pre-motor cortex, pre-frontal cortex, and orbitofrontal cortex.