Distributed fading memory for stimulus properties in the primary visual cortex

Abstract

It is currently not known how distributed neuronal responses in early visual areas carry stimulus-related information. We made multielectrode recordings from cat primary visual cortex and applied methods from machine learning in order to analyze the temporal evolution of stimulus-related information in the spiking activity of large ensembles of around 100 neurons. We used sequences of up to three different visual stimuli (letters of the alphabet) presented for 100 ms and with intervals of 100 ms or larger. Most of the information about visual stimuli extractable by sophisticated methods of machine learning, i.e., support vector machines with nonlinear kernel functions, was also extractable by simple linear classification such as can be achieved by individual neurons. New stimuli did not erase information about previous stimuli. The responses to the most recent stimulus contained about equal amounts of information about both this and the preceding stimulus. This information was encoded both in the discharge rates (response amplitudes) of the ensemble of neurons and, when using short time constants for integration (e.g., 20 ms), in the precise timing of individual spikes (<or= approximately 20 ms), and persisted for several 100 ms beyond the offset of stimuli. The results indicate that the network from which we recorded is endowed with fading memory and is capable of performing online computations utilizing information about temporally sequential stimuli. This result challenges models assuming frame-by-frame analyses of sequential inputs.

Figure 1. Experimental setup and illustration of the analysis method.

(A) An example of a visual stimulus in relation to the constellation of receptive fields (rectangles) from one Michigan probe. (B) Upper part: spike times recorded from one neuron across 50 stimulus presentations and for two stimulus sequences (ABC and DBC). In this and in all other figures, the gray boxes indicate the periods during which the letter stimuli were visible on the screen. Lower part: peristimulus time histogram (PSTH) for the responses of this neuron (5-ms bin size). (C) Left: spike trains obtained simultaneously from 66 neurons in one stimulation trial. Blue: the neuron for which all 50 trials are shown in (B). Right: for the classification analysis, each spike train is convolved with an exponential kernel (i.e., low-pass filtered). The spike trains for only six example neurons are shown. Red: example values of the convolved trace that are used as inputs to the classifier (far right).

Figure 2. The ability of a linear classifier to determine the identity of the presented letters, A or D.

The classification performance is shown (solid line) as a function of time passed from the presentation of the stimulus (at 0 ms) until the moment at which a sample of neuronal activity was taken for training/testing the R_t classifier (the stimulus was removed at 100 ms). Dash-dotted line: the mean firing rate across the entire population of investigated neurons. Dotted line: expected performance at chance level (50% correct). The shaded horizontal stripe around the dotted line covers the region of statistically nonsignificant deviations from the chance level (p>0.05), estimated by a label-shuffling test. In this and all other figures, the color of the stripe matches the color of the performance curve for which the statistical test was performed. (A and B) Two different experiments on different cats.

Figure 3. Availability of information about a stimulus presented as a part of a sequence.

Classifiers R_t were trained to identify the first letter in the sequences, i.e., ABC versus DBC in one experiment (cat 1) and ABE versus CBE in the other two experiments (cats 2 and 3). (A–C) Three different experiments on different cats. Notations are the same as described in Figure 2.

Figure 4. Simultaneous availability of information about multiple stimuli in a sequence.

(A) Performance of time-specialized classifiers R_t trained on individual time points to identify the second letter in the sequences of three letters (i.e., ABC vs. ADC). The results should be compared to Figure 3A. (B and C) Simultaneous availability of information about two different letters of a sequence. The following four sequences were presented: ABE, CBE, ADE, and CDE, and two classifiers identified each the presentation of either the first (blue line) or the second letter (green line). Shaded stripe and dotted lines are as described in Figure 2. (D and E) R_t classification performance for 16 different recordings made across four cats. Due to the sparseness of responses, the results are given as average classification performance within 100-ms windows. Thick black curve: gross average across all datasets. Vertical lines: standard error of the mean across all datasets of all cats. Asterisk: the datasets chosen for further analysis.

Figure 5. The change in classification performance for R_t classifiers as a function of the time constant, τ.

(A) τ is changed systematically between 1 and 100 ms (both for training and testing). The performance reached its plateau at about 20 ms. (B) Detrimental effect of 10-ms jitter on the classification performance achieved with different values of τ. As a rule, jitter caused a stronger drop in performance with small than with large values of τ. However, this function was not monotonic, as the performance drop was strongest when τ had values of 5–10 ms. Vertical bars: standard error of the mean across jittered datasets. (C–E) A detailed analysis of the relation between the value chosen for the integration constant, τ (1…100 ms) and the classification performance of R_t readouts. Changes in performance are shown for three cats. As a rule, classification performance increases with longer integration constants.

Figure 6. The performance of the classifiers R_t (from Figure 3) as a function of Gaussian jitter applied to spike timing.

(A) Change in classification performance as a function of the amount of jitter shown for the three points with the highest classification prior to the application of jitter. The results are shown for three different points in time exhibiting the strongest effects (cat 1: 60,120, and 200 ms; cat 3: 40, 120, and 230 ms). (B) The same analysis as in (A) but for spike trains shuffled across presentation trials (cat 1: 40, 130, and 220 ms; cat 3: 40, 120, and 250 ms). Vertical lines: standard error of the mean across all jittered and trial-shuffled datasets.

Figure 7. Performance of classifiers R_int trained to extract information during an interval lasting 100–300 ms (τ = 20 ms).

(A) Performance for four linear classifiers, each trained for one of the four consecutive intervals of 100-ms duration (color-coded bars). Thin gray line: classification performance of specialized R_t classifiers. (B) The weight vectors learned by the four classifiers in (A). (C–E) The performance of SVM classifiers with different kernels trained on 200-ms (D) and 300-ms (C and E) intervals for three different experiments. Shaded stripe and dotted line are as described in Figure 2.

Figure 8. Information contents of neuronal firing rates and their second-order correlations.

(A) Left y-axis: the average firing rates (Mfrs) in response to two different stimuli for the four most informative units in the analysis of time invariant classification with int = 150–450 ms, cat 1. Right y-axis: area under receiver-operating characteristic (AUC), related to the probability of correct classification. (B) Pairwise correlations in rate responses, expressed as a product between Mfrs in (A) and the corresponding AUC values obtained from these products.