Temporal context and conditional associative learning

Abstract

Background: We investigated how temporal context affects the learning of arbitrary visuo-motor associations. Human observers viewed highly distinguishable, fractal objects and learned to choose for each object the one motor response (of four) that was rewarded. Some objects were consistently preceded by specific other objects, while other objects lacked this task-irrelevant but predictive context.

Results: The results of five experiments showed that predictive context consistently and significantly accelerated associative learning. A simple model of reinforcement learning, in which three successive objects informed response selection, reproduced our behavioral results.

Conclusions: Our results imply that not just the representation of a current event, but also the representations of past events, are reinforced during conditional associative learning. In addition, these findings are broadly consistent with the prediction of attractor network models of associative learning and their prophecy of a persistent representation of past objects.

Figure 1

Experimental design (schematic). Each trial comprises three phases: stimulus presentation, motor response, and reinforcement. Firstly, a fractal object appears (center), surrounded by four response options (grey discs). Secondly, the observer reacts by pressing the key that corresponds to one response option (outlined disk). Thirdly, a color change of the chosen option provides reinforcement (green if correct, red if incorrect). (b) Object sequence with temporal context. Target objects recur every 2 to 48 trials. Thus, successive trials always present different objects. A consistent temporal context is created by the fact that each target object (e.g., trials t and t + m) is preceded consistently by a specific (other) object (trials t - 1 and t + m - 1). (c) Object sequence without temporal context. Each time an object appears (trials t and t + m), it is preceded by a different object (trials t - 1 and t + m - 1).

Figure 2

Behavioral and modeling results. For each of five experiments, temporal context, behavioral performance, and predicted performance are shown (left, middle, and right columns, respectively). Trial sequences were composed of 'recurring objects' (types A-F) distinguished by their temporal context. Error bars refer to the 95% confidence intervals (α = 0.05) for binomially distributed data. In (b)-(e), recurring objects were intermixed with 'one-time objects'. Type A objects were preceded by a one-time object and followed by one particular other recurring object (probability 100%). Type B objects were preceded by one particular other recurring object (probability 100%) and followed by a one-time object. Type C objects were preceded (followed) by one-time objects (probability 50%) and by each of several other recurring objects (cumulative probability 50%). Type D objects were preceded (followed) by one-time objects (probability 50%) and by one particular other recurring object (probability 50%). Type E objects were preceded by a one-time object and followed by each of four other recurring objects (probability 25%). Type F objects were preceded by each of four other recurring objects (probability 25%) and followed a one-time object. The relative informativeness of the temporal contexts is given in Table 1. (a) Eight objects appeared seven times each, in either deterministic or random sequences. In deterministic sequences, each object was preceded (followed) seven times (100% probability) by one particular of the other seven objects. In random sequences, each object was preceded (followed) once (14% probability) by each of the seven other objects. (b) Eight recurring objects (2 type A, 2 type B, and 4 type C) appeared six times each, intermixed with one-time objects. (c) Sixteen recurring objects (4 type A, 4 type B, and 8 type C) appeared 14 times each. (d) Ten recurring objects (5 type C and 5 type D) appeared eight times each. (e) Sixteen recurring objects (4 each of types A, B, E, and F) appeared 8 times each.

Figure 3

Actual learning rates and estimated parameters. (a) Acceleration of learning during the initial appearance of objects due to different degrees of temporal context. In the presence of a fully predictive temporal context, the accumulation of information was accelerated by 0.13 bit. Error bars in Figure 3a show the standard deviation across experiments for each object type. Plots (b)-(f) show regions of optimal values in the parameter space (ϵ, γ), corresponding to the general learning rate and the recognition parameter, respectively. The color scales to the right of each plot refer to the fit quality f_Q for each parameter pair (ϵ, γ), which was computed as , where and are the mean values of performance correct in the i-th appearance for human observers and for the model simulations, respectively, and and are the corresponding standard deviations. The higher the f_Q values, the better the fit between measured and predicted data.

Figure 4

Reinforcement of action values (schematic). Each object is associated with 12 action values. For the object in trial t, 4 action values inform the response of the current trial t, 4 values concern the response of the next trial t + 1, and the remaining 4 values contribute to the response of the second next trial t + 2. Correspondingly, the response of trial t is based on 12 actions values: 4 values of the current object t, 4 values of the previous object t - 1, and 4 values of the pre-previous object t - 2. Temporal context determines which action values are reinforced consistently. (a) In the absence of temporal context, only the current object's action values are reinforced consistently and come to reflect the correct choice. In this case, the decision in trial t is based on 4 action values of object t. (b) In the presence of temporal context, both the current and the previous object's action values are reinforced consistently. Thus, the decision in trial t is based on 4 action values of object t and 4 action values of object t - 1.