Characterizing the effects of feature salience and top-down attention in the early visual system

Abstract

The visual system employs a sophisticated balance of attentional mechanisms: salient stimuli are prioritized for visual processing, yet observers can also ignore such stimuli when their goals require directing attention elsewhere. A powerful determinant of visual salience is local feature contrast: if a local region differs from its immediate surround along one or more feature dimensions, it will appear more salient. We used high-resolution functional MRI (fMRI) at 7T to characterize the modulatory effects of bottom-up salience and top-down voluntary attention within multiple sites along the early visual pathway, including visual areas V1-V4 and the lateral geniculate nucleus (LGN). Observers viewed arrays of spatially distributed gratings, where one of the gratings immediately to the left or right of fixation differed from all other items in orientation or motion direction, making it salient. To investigate the effects of directed attention, observers were cued to attend to the grating to the left or right of fixation, which was either salient or nonsalient. Results revealed reliable additive effects of top-down attention and stimulus-driven salience throughout visual areas V1-hV4. In comparison, the LGN exhibited significant attentional enhancement but was not reliably modulated by orientation- or motion-defined salience. Our findings indicate that top-down effects of spatial attention can influence visual processing at the earliest possible site along the visual pathway, including the LGN, whereas the processing of orientation- and motion-driven salience primarily involves feature-selective interactions that take place in early cortical visual areas.NEW & NOTEWORTHY While spatial attention allows for specific, goal-driven enhancement of stimuli, salient items outside of the current focus of attention must also be prioritized. We used 7T fMRI to compare salience and spatial attentional enhancement along the early visual hierarchy. We report additive effects of attention and bottom-up salience in early visual areas, suggesting that salience enhancement is not contingent on the observer's attentional state.

Keywords: fMRI; lateral geniculate nucleus; primary visual cortex; salience; visual attention.

Fig. 1.

Annotated sample displays for both experiments. Feature-contrast salience is defined by orientation in A and by drifting motion direction in B (indicated by arrows, which were not present during the experiment). Dashed circles (likewise not present during the experiments) indicate the 2 locations at which the salient patch could appear and to which spatial attention could be directed. Although the task-relevant contrast decrement occurred at both of these locations, the observer was instructed to only perform the detection task on 1 patch, as indicated throughout each block by a small cue. In these examples, if the participant is cued to attend to the black cue (as labeled), in A, the left patch is attended/salient, whereas the right patch is unattended/nonsalient; in B, the left patch is attended/salient, and the right patch is unattended/salient. Gabor patch edges were Gaussian blurred and also spatially separated by a gap of 0.8°, yielding an effective spatial separation of ~1.8°. dva, Degrees of visual angle.

Fig. 2.

Results of experiment 1 for the 4 salience and attention conditions. A: magnitude of the attention and salience effects across ROIs, computed as the difference in percent signal change. Circles show the effect in individual subjects, and error bars depict SE (across subjects). In the LGN, only attention significantly modulated BOLD responses; neither the main effect of salience nor the interaction effect were significant. A significant effect of salience first emerged in V1 and is evident in each cortical ROI. Attention also modulated BOLD responses in V1 through hV4, but the 2 effects did not reliably interact in any region. B, top: mean ROI time courses time-locked to the beginning of each experimental block, which lasted 8 TRs and is demarcated by dotted lines. Bottom, the same data averaged across the block, offset to account for hemodynamic lag, and normalized by subtracting the mean response of the 2 TRs immediately preceding the block. Error bars on block averages depict SE in the mean BOLD response across subjects.

Fig. 3.

Examples of average fMRI time courses from representative subjects in experiment 1 (A and B) and experiment 2 (C and D). Error bars depict ±SE for each experimental condition, and each of the 4 panels shows data from a different individual. As expected, there is some variability between subjects; fMRI responses in the LGN are also more variable than those in early visual areas, partly because of the LGN’s smaller size and the presence of greater physiological noise in midbrain structures.

Fig. 4.

Magnitude of attention and salience effects plotted as a function of ROI size. We calculated the difference in BOLD response for attended minus unattended conditions and salient minus nonsalient conditions for a wide range of ROI sizes, ranging from 2 to 60 maximum voxels per hemisphere in the LGN and from 10 to 350 voxels per hemisphere in individual cortical visual areas. Error bars indicate SE across subjects. Our findings of positive salience and attention effects in each cortical ROI are highly consistent across a wide range of ROI sizes.

Fig. 5.

Mean BOLD amplitudes in experiment 2, in which salience was defined by the direction of drifting motion of Gabor patches. A: attention and salience effects (% signal change difference) across ROIs, with circles plotting individual subjects’ results and error bars showing SE across subjects. B: results in each condition averaged across the stimulus block and normalized by a prestimulus window for each condition. The pattern of results follows that of experiment 1: attentional enhancement is evident in each ROI, including the LGN, whereas salience modulated only cortical mean BOLD responses. In every region studied, the effects of attention and salience did not significantly interact.

Fig. 6.

Individual subject ANOVA results for experiment 1 (Expt 1; left) and experiment 2 (Expt 2; right). Each experimental session was analyzed independently, with experimental run as the repeated measure. Filled circles indicate significant effects of attention (black), salience (gray), or their interaction (cross) at P < 0.05 level. Subject labels (S1–S6) are arbitrary and unmatched between the 2 experiments. Although many subjects exhibited significant main effects of attention and salience, the interaction of these 2 factors was not significant in any ROI in any subject.