Impact of spatiotemporally correlated images on the structure of memory

Abstract

How does experience modify what we store in long-term memory? Is it an effect of unattended experience or does it require supervision? What role is played by temporal correlations in the input stream? We present a plastic recurrent network in which memory of faces is initially embedded and then, in the absence of supervision, the presentation of temporally correlated faces drastically changes long-term memory. We model and interpret the results of recent experiments and provide predictions for future testing. The stimuli are frames of a morphing film, interpolating between two memorized faces: If the temporal order of presentation of the frame stimuli is random, then the structure of memory is basically unaffected by synaptic plasticity (memory preservation). If the temporal order is sequential, then all image frames are classified as the same memory (memory collapse). The empirical findings are reproduced in the simulated dynamics of the network, in which the evolution of neural activity is conditioned by the associated synaptic plasticity (learning). The results are captured by theoretical analysis, which leads to predictions concerning the critical parameters of the stimuli; a third phase is identified in which memory is erased (forgetting).

Fig. 1.

The network, stimuli, and dynamics. (a) Scheme of the network. Circles, eight neurons (N = 1,000 neurons in simulations); two-headed arrows, recurrent synaptic connections connecting two neurons of the network; one-headed arrows, external connection (input stimulus); rectangles, tuning curves (sigmoidal) of the neurons along the morphing sequence, i.e., external current vs. frame stimulus index; vertical bar in each rectangle, index α of a sample frame stimulus. (b) The state of each neuron [+1, (red) or −1 (blue)] vs. frame number r along the morphing sequence. r = 1, network state F, has the highest number of neurons in −1 state (blue); r = 30: NF, has highest number of +1's (red). Neurons (features) are switched monotonically, proceeding from F to NF. (c) External current (color-coded) to all neurons vs. frame index α. Neurons are ordered vertically by increasing values of the neuron index η. The black line corresponds to zero current. (d) Sample network dynamics. The first 10 trials of a simulation (mixed protocol, c = 0.4, T = 0.5) are presented as the total current received by each neuron vs. time. For each presented frame r, the first column is the stimulation interval, and the second column is the corresponding delay interval. Values of the current are color-coded. The Black curve is at zero current (η = η*). Neurons above this curve are −1, and neurons below are +1. Two (delay) attractor states (η* = ±0.35) can be observed.

Fig. 2.

Network dynamics in the mixed protocol. Total current (color-coded) afferent to each neuron (labeled η) vs. time for 30 trials (one session). Each trial consists of stimulus + delay interval. Abscissa is the frame number (r) of each stimulus. The black curve is the label η* of the neuron receiving zero current. In each interval, stimulus or delay, η* fluctuates around a fixed value, respectively η_s* and η_d*. (a) Session 1. Two delay activity states are present at η_d* ≈ −0.35 (F response) and η_d* ≈ +0.35 (NF response). Visual responses vary from frame to frame. (b–d): Sessions 4 (η_d* ≈ ±0.29) (b), 7 (η_d* ≈ ±0.25) (c), and 10 (η_d* ≈ ±0.22). Stimuli from the first (second) half, i.e., r ≤ 15 (r > 15), result in F (NF) responses. (T = 0.5; c = 0.4).

Fig. 3.

Delay activity and visual response labels in 10 sessions (300 trials) of the simulations in each of the two protocols. (a) η_d*, label of delay activity states (average over fluctuations in the corresponding interval ± SD) vs. trial number in the mixed protocol. Curves, theoretical predictions; solid lines, labels of stable states; dashed line, label of unstable states. The two solid lines correspond to the two memory states, determining F and NF responses, in agreement with simulation results. The dashed line (η_d* = 0) represents the unstable state, the watershed between the two memory attractors. The two memory states are preserved. (b) η_s*, visual response label (average ± SD) vs. trial number in the mixed protocol. The format is the same as in a. Labels of visual response states cluster around the memory states. (c) Labels of delay activity states vs. trial number in the sequential protocol. The format is the same as in a. From session to session (30 trials each), the number of F responses increased, as is evidenced by the fact that the number of delay labels near the upper curve diminishes and that near the bottom curve they increase. Beyond trial 180, there are no NF responses. Next, the line of NF delay activity annihilates in a saddle node. (d) Labels of visual response states vs. trial number in the sequential protocol. Like memory states, visual responses tend to leave the NF line and move toward F until all stimuli evoke a state near F. (T = 0.5, c = 0.4).

Fig. 4.

Network dynamics in the sequential protocol (format is the same as in Fig. 3). Frame stimuli are presented sequentially (r = 1.,,30, abscissa). (a) Session 1. The transition of responses from NF (positive η_d*) to F (negative η_d*) at r = 18 is shown. (b) Session 4. Transition at r = 20 is shown. (c) Session 7. All frames are classified F, but η_s* > 0, for r approaching 30. (d) Session 10. All η_s* < 0, i.e., both visual responses and delay states close to the F state. (T = 0.5, c = 0.4).

Fig. 5.

Forgetting regime. Network dynamics in the mixed protocol, for T/c = 0.6 is shown. (a and b) Total current (format is the same as in Fig. 2). (a) Session 1. This is very similar to Fig. 2a (T/c = 1.25), except for the more widely distributed visual responses. (b) Session 10. For all delay states, η_d* ≈ 0, i.e., equal numbers of blue and red neurons, hence random responses. (c) η_d* of delay activity states vs. trial number (format is the same as in Fig. 3). Lines are theoretical predictions for delay activity label. Solid lines, stable; dashed line, unstable. In contrast to the case T/c = 1.25 (Fig. 3a), the two memory states are not preserved; they collapse onto a single stable state after 275 trials. (d) Scatter plot of η_s* of visual response states vs. trial number. Visual responses are broadly distributed (compare with Fig. 3b for T/c = 1.25). (T = 0.3; c = 0.5).