Effects of temporal context and temporal expectancy on neural activity in inferior temporal cortex

Abstract

Timing is critical. The same event can mean different things at different times and some events are more likely to occur at one time than another. We used a cued visual classification task to evaluate how changes in temporal context affect neural responses in inferior temporal cortex, an extrastriate visual area known to be involved in object processing. On each trial a first image cued a temporal delay before a second target image appeared. The animal's task was to classify the second image by pressing one of two buttons previously associated with that target. All images were used as both cues and targets. Whether an image cued a delay time or signaled a button press depended entirely upon whether it was the first or second picture in a trial. This paradigm allowed us to compare inferior temporal cortex neural activity to the same image subdivided by temporal context and expectation. Neuronal spiking was more robust and visually evoked local field potentials (LFP's) larger for target presentations than for cue presentations. On invalidly cued trials, when targets appeared unexpectedly early, the magnitude of the evoked LFP was reduced and delayed and neuronal spiking was attenuated. Spike field coherence increased in the beta-gamma frequency range for expected targets. In conclusion, different neural responses in higher order ventral visual cortex may occur for the same visual image based on manipulations of temporal attention.

Figure 1

Recording Location. On a lateral view of the macaque brain (A.), the vertical black line shows the dorsal to ventral approach used to reach visually responsive inferior temporal cortex (gray). B. An outline of a coronal section (adapted from (Paxinos, Huang, & Toga, 2000) of the recording plane with the same gray region highlighted. The recording area targeted the inferior lip of the superior temporal sulcus and the deeper cortex on the lateral temporal lobe

Figure 2

Behavioral Task Overview. A) The time course of a single trial. Trials began with a centered fixation spot. After the monkey fixated the spot, it was extinguished, and a single visual image from a library of 100 appeared centered in a dynamic colored frame (cue). After 500 msec this image was extinguished, but the dynamic frame remained. A delay of 1000 msec (early trials) or 2000 msec (late trials) followed after which a second image (target) selected from the same set of 100 images appeared in the center of the colored frame. The animal had to press a button to acquire a juice reward. Each image in the set of 100 was associated with both a specific temporal delay (1000 or 2000 msec) and a specific button press (right or left), each randomly assigned. The animal learned these associations over several months of preliminary behavioral training. B) Trial types used in the experiments. For 1/5^th of the trials the delay value of the cue was invalid. For example, the pacifier was a “late” cue and was followed 4/5 of the time by a 2000 msec delay. However, for invalid trials the target would appear unexpectedly early (“Invalid cue/Early target” --- row 3). In addition, the pacifier could also appear as a target (as shown in row 4).

Figure 3

Response Times by Delay, Target Contrast, and Cue Validity. The black bars in the center of the boxes mark the median response times. Boxes show the 25 and 75% range and the 10 and 90% ranges are delimited by the whiskers. For each contrast value the invalid early trials (lighter shaded boxes, upper panel) are, on average, slower. Lower contrast images are responded to more slowly.

Figure 4

Neural Effects of Meaning. A) The raster plots for one neuron along with the spike density functions are presented divided into cue presentations and target presentations. The target trials are those when the image was validly cued and full contrast. Note the decreased number of spikes to cue presentations. B) The spike density functions for the full population of recorded cells are shown and demonstrate the consistent increases for target trials across the population. C) Shows the visually evoked LFP for the population. Note an increase in magnitude of the signal for the target presentations. D) This quantifies the probability of the difference between the two LFP traces shown in panel C. We subtracted the average LFP for target presentations from that for cue presentations (heavy blue trace). We then randomized the labels: target or cue, and repeated the average and differencing 1,000 times. The thinner lines mark the 99% confidence intervals computed from this randomization procedure. The time window when there were consistent differences in the magnitude of the LFP for cue and target presentations is marked by vertical lines.

Figure 5

Effects of Temporal Attention on Spike Rate - Qualitative. This figure provides an overview of the effect of temporal attention on spike rates. Spike density functions are shown collapsed across data sets and monkeys for all early and late, validly and invalidly cued, trials where the target appeared at full contrast. Validly cued trials are shown by solid lines and invalidly cued trials by broken lines. Since there were only two time points at which images could appear, only the early, invalidly cued, trials should be unexpected. This is confirmed by the similarity of all the spike density functions except for early, invalidly cued trials which show a reduced initial transient response and a greater number of later spikes. Because some neuronal populations show delay activity, we extend the display to 450 msec before image onset (target onset occurred at time 0). There was no increase in spiking during the delay period, nor were there any differences across conditions.

Figure 6

Effects of Temporal Attention - Detailed. This figure shows the spike density functions for neurons and evoked potentials for all contrasts (columns) and delays (rows) subdivided by cue validity. To examine the temporal evolution of these differences and to demonstrate their statistical significance we calculated a p-value for sliding data windows from -50 to 450 msec after image onset. The scale for the p-values is on the right side of each graph and is a base 2 logarithmic scale where the increment of each tick mark reflects a doubling/halving from 0.5. The upper two rows of plots show that there are consistent statistical increases in early spike numbers for validly cued early trials. The bottom two rows show analogous findings for the visually evoked LFP.

Figure 7

LFP Power Increases in the Beta Frequency Range for Expected Targets. The LFP power at each frequency band between DC and 80 Hz was computed for the time period 525 - 1025 msec after cue image offset and figure shows the ratio of power for the 1000 msec cue trials relative to the power for the 2000 msec cue value trials. The binomial distribution was used to calculate the probability of the fraction of neurons showing values greater than 1.0. The box and whisker plot shows the median difference value as a black line, the boxes demarcate the 25 to 75% range and the whiskers mark the 10 to 90% range. Frequency bands with a significance of p < 0.001 (binomial) are colored black. There is a broad band from 18 - 24 Hz (beta frequency range) where there is consistently greater power for trials where the expected target is imminent.

Figure 8

Spike Field Coherence. Panels A and B show the SFC for 1000 msec trials (left) and 2000 msec trials (right). Invalid trials are shown as dashed lines. Below each SFC plot we show the median (black bar) and 99% confidence intervals (gray boxes) for the difference at each frequency band between the valid and invalid conditions normalized by the SFC for the validly cued trials. The confidence intervals were computed from a bootstrap resampling with 1000 repetitions. Both target delays show changes at the low frequencies, but only the 1000 msec comparison (Panel C) shows a broad increase in the beta to gamma frequency range (32 - 40 Hz). Increases in low frequency SFC are seen for both trial types and may reflect an error signal.

Figure 9

Eye Position Distributions. A. For each dataset the location of the eyes from the center of the screen at the time of image onset was calculated and the mean of the invalid trials was subtracted from the mean of the valid trials. For both trial types the difference was less than 0.5 degrees of visual angle. For early trials the eyes tended to be slightly farther from the center of the screen on invalid trials compared to valid trials (the opposite was observed for late trials), but the distance was much smaller than the size of IT visual neurons’ receptive fields. B. We created two groups of near and far trials by performing a median split of all valid trials and then compared the mean number of spikes per trial in these two groups. The inset shows that the median difference was zero, and therefore slight differences of the eyes with respect to the center of the screen do not explain the differences in spiking and LFP reported above.