Attention and non-retinotopic feature integration

Abstract

Features of moving objects are non-retinotopically integrated along their motion trajectories as demonstrated by a variety of recent studies. The mechanisms of non-retinotopic feature integration are largely unknown. Here, we investigated the role of attention in non-retinotopic feature integration by using the sequential metacontrast paradigm. A central line was offset either to the left or right. A sequence of flanking lines followed eliciting the percept of two diverging motion streams. Although the central line was invisible, its offset was perceived within the streams. Observers attended to one stream. If an offset was introduced to one of the flanking lines in the attended stream, this offset integrated with the central line offset. No integration occurred when the offset was in the non-attended stream. Here, we manipulated the allocation of attention by using an auditory cueing paradigm. First, we show that mandatory non-retinotopic integration occurred even when the cue came long after the motion sequence. Second, we used more than two streams of which two could merge. Offsets in different streams were integrated when the streams merged. However, offsets of one stream were not integrated when this stream had to be ignored. We propose a hierarchical two stage model, in which motion grouping determines mandatory feature integration while attention selects motion streams for optional feature integration.

Figure 1

The sequential metacontrast paradigm. (a) Stimulus. A central line is followed by four consecutive pairs of flanking lines. The central line is offset either to the left (not shown) or to the right (as shown here). All flanking lines are non-offset. (b) Percept. Observers perceive two diverging streams of lines, one moving to the left and one to the right. The central line itself is invisible. Still, its offset can be perceived at the flanking lines (Otto et al., 2006; see Movie 1 for an animation).

Figure 2

(a) Motion grouping. Single lines appear as one line in apparent motion. Without motion grouping, a sequence of separate lines would be perceived. (b) Feature attribution. The offset of the first line is attributed to the moving line, which is perceived offset even though physically the later lines are aligned. (c) Feature integration. The right- and the left-offset of two lines are integrated in a manner analogous to mathematical integration (summation) and cancel out each other. (d) Attention may access the offsets of the single lines (dotted line) or only the integrated offset (solid line). In the former case, integration is optional, in the latter, it is mandatory.

Figure 3

Task and offset conditions. (a) Auditory cue. A beep indicated whether the leftward or the rightward motion stream had to be attended. We asked observers to report the offset of the cued stream. Observers pressed the left/right button if the lower line segment was displaced to the left/right with respect to the upper segment. Block-by-block, the beep occurred either before, during, or after the two motion streams were presented. (b) Offset conditions. In condition C, only the central line was offset (central-offset). In condition F, the third flanking line in the cued stream was offset (flank-offset). In condition CF, both the central- and the flank offset were presented. The direction of the central-offset was always in the direction of a reference (left or right, randomly chosen on each trial). The flank-offset was always in the opposite direction as the reference. Hence, when the central-offset was to the right, the flank-offset was to the left, and vice versa. All six combinations of cued stream (leftward, rightward) and offset condition (C, F, CF) were randomized within a block of trials. (c) Lapse control. Randomly in half of the trials of the conditions shown in b, we presented an additional offset in the non-cued stream (the central- and/or flank-offset are not shown here). The direction of this probe-offset was randomly in the same or in the opposite direction as the reference. If the probe-offset contributes to performance (i.e., when lapses have occurred), we expect a systematic difference in performance for “same” compared to “opposite” probe trials. If observers report selectively the offset of the cued stream, we expect not such a difference. Offset lines are highlighted in black, actual stimuli were bluish white on a dark background.

Figure 4

Cued attention. In all panels, the vertical gray band indicates the duration of the motion streams. Green and red bands indicate the duration of the central- and the flank-offset lines, respectively (Figure 3b). (a) Performance in conditions C and F as a function of cue-stimulus onset asynchrony (CSOA). The sign of the dominance level indicates whether the central- (+) or the flank-offset (−) dominates the responses. The absolute value reflects the strength of the corresponding dominance. For a CSOA of −500 ms, performance was similar to conditions when observers attended always the same stream, i.e. in non-cued conditions (dotted lines). With increasing CSOA, dominance decreased slightly but similarly for conditions C and F. (b) Performance in condition CF. For all CSOAs, performance was around 0%. The actual dominance level was well predicted by the sum of the dominance levels achieved in conditions C and F (C + F). Performance was similar to when always the same stream was attended in a block of trials (dotted lines). (c) Lapses control. With the probe-offset in the non-cued stream, performance differs only slightly in same/opposite probe-offset trials (Figure 3c). This difference is close to the difference observed in a non-cued comparison condition (thin dotted lines) and much smaller compared to a comparison condition in which the stream with the probe-offset was attended (thin solid lines). (d) Reaction times. With a CSOA of −500 ms, reaction times were slightly longer than with one attended stream only (dotted lines). As the CSOA increased, reaction times increased. There was virtually no difference in reaction times for conditions C, F, and CF. Means and SEM for 6 observers.

Figure 5

Control experiment. An auditory cue indicated whether a left (L) or a right static line (R) had to be attended. The cued line was offset in the same direction as the reference. The non-cued line was offset randomly in the same (as in R) or in the opposite direction as the reference (as in L). We varied the onset of the cue with respect to the onset of the lines (on-screen as indicated by the gray band). (a) Performance. With a CSOA of −500 ms, the dominance level was similar to conditions when observers attended always to the same line in a block of trials (dotted line). Dominance decreased slightly with increasing CSOA. For all CSOAs, we found no performance differences whether the left or right line was cued (data not shown). The same held true when we compared trials with the non-cued line in the same or in the opposite direction as the reference (data not shown). (b) Reaction times increased the later the cue was presented. Mean and SEM for 6 observers.

Figure 6

Merging motion streams. (a) Stimulus. We presented two sequential metacontrast sequences simultaneously and next to each other. The two sequences converged onto a central flanking line in the last frame. (b) Percept. The visibility of the two “central lines” is strongly suppressed. Observers perceive four motion streams with the two inner streams merging in the central line (for an animation, see Movie 2). (c) Offset conditions. Randomly interleaved, the left (A), the right (B), or both lines in the first frame (AB) were offset. Offset A was always in the same direction as the randomly chosen reference. Offset B was always in the opposite direction as the reference. Hence, if offset A was to the right, offset B was to the left, and vice versa. All flanking lines were non-offset.

Figure 7

Observers attended to the last lines in the stream. (a) When the center line was attended, both offset A and B contributed to performance. In condition AB, the dominance level was strongly reduced indicating an integration of the two offsets (with a slight dominance of offset A). (b) If the left line was attended, offset A dominated in conditions A and AB. Offset B did not, or only marginally, contribute to performance. Analogous results held true when the right line was attended. (c) We removed the last frame of the sequence. Hence, the two inner motion streams terminated before they merged. We asked observers to indicate the offset of the central left (Center-L) or the central right line (Center-R). No integration occurred similar to (b). Mean and SEM of 5 observers.

Figure 8

(a) We repeated the experiment with attention to the central line. Results are similar to Figure 7a. (b) We asked observers to focus on the stream coming from the left (Center-L) or right (Center-R). When the motion stream coming from the left was attended, offset A dominated and offset B contributed only marginally. When the stream coming from the right was attended, we found the inverse. (c) Post-cueing. An auditory cue with a CSOA of 500 ms indicated whether the stream coming from the left or right had to be attended. Performance is very similar to (b) except for a small decay in performance like in Experiment 1. Mean and SEM of 5 observers.