Inversion of pop-out for a distracting feature dimension in monkey visual cortex

Abstract

During visual search, it is important to reduce the interference of distracting objects in the scene. The neuronal responses elicited by the search target stimulus are typically enhanced. However, it is equally important to suppress the representations of distracting stimuli, especially if they are salient and capture attention. We trained monkeys to make an eye movement to a unique "pop-out" shape stimulus among an array of distracting stimuli. One of these distractors had a salient color that varied across trials and differed from the color of the other stimuli, causing it to also pop-out. The monkeys were able to select the pop-out shape target with high accuracy and actively avoided the pop-out color distractor. This behavioral pattern was reflected in the activity of neurons in area V4. Responses to the shape targets were enhanced, while the activity evoked by the pop-out color distractor was only briefly enhanced, directly followed by a sustained period of pronounced suppression. These behavioral and neuronal results demonstrate a cortical selection mechanism that rapidly inverts a pop-out signal to "pop-in" for an entire feature dimension thereby facilitating goal-directed visual search in the presence of salient distractors.

Keywords: V4; enhancement; monkey; suppression; visual search.

Fig. 1.

Task description and behavioral results. (A) Real-life example of visual search with a salient distractor. When looking for your keys on a crowded desk, you may be looking for small key-shaped objects. Your attention may however be captured by salient objects like the bright green parrot, which might interfere with the process of finding your keys. (B) We recorded from area V4 while monkeys performed a visual search task in which they selected the odd-shape-out (here a square among circles) with an eye movement. One of the six visual items was in the V4 receptive field. The target was the stimulus that differed from the others by shape. Nonsalient distractor stimuli had the same color as the target, while a single salient distractor stimulus popped out because it had a different color. (C) Example series of three trials. In the second trial, the target and distractor shapes swapped with respect to the first trial (this occurred 50% of the time). In the third trial, the target and distractor colors swapped (this also occurred 50% of the time). In addition, the reward magnitude was randomly varied (50% high, 50% low). (D) Accuracy (green bars) and the proportion of trials on which the monkeys made an error by choosing a nonsalient distractor (ND, gray bars) or the salient distractor (SD, red bars). Nonsalient distractors are four times more prevalent than targets and salient distractors (prevalence indicated with dashed horizontal lines). The Insets show the proportion of choices of distractor stimuli corrected for prevalence. Even after this correction, the animals chose the salient distractor less often than the nonsalient distractors (* indicates P < 0.001 for a one-tailed t test SD < ND). Error bars indicate the SD over recording sessions. (E) The effects of color and shape swaps on accuracy (Top) and reaction time (Bottom) for both monkeys. Yellow lines indicate trials in which the target and salient distractor colors swapped relative to the previous trial; blue lines are trials in which those colors stayed the same. The horizontal axis indicates whether the target shape changed relative to the previous trial. Error bars (often smaller than the data points) indicate SEM, asterisks denote P < 0.001 for main effects as indicated by two-way analyses of variance (ANOVAs; no interaction effects were significant at P < 0.05). (F) Dependence of erroneous choices on the relative locations of the target (T) or salient distractor (SD) stimuli. The proportion of SD or ND choices on error trials is plotted as function of the distance between the chosen stimulus in the search array (a distance of one indicates the two stimuli were next to each other, a distance of two means there was one stimulus in between, etc.), the identity of the chosen stimulus (gray: ND; red: SD), and the reaction time (30% fastest and slowest response indicated with square and diamond symbols respectively). The dashed lines indicate chance level.

Fig. 2.

Saccadic reaction times and choices. (A) Distributions of shortest saccadic reaction times (SRTs, fastest 25th percentile) for target (T, green) and salient distractor choices (SD, red) in the two monkeys. The distributions were normalized such that both the red and green bars sum up to 100% (see SI Appendix, Fig. S2 for the full SRT distributions, normalized within choice type (as here) and also by the total number of saccades). The dark colors indicate overlap between the red and green distributions. The probability of choosing the salient distractor was increased at short SRTs (black arrows). (B) Proportion of salient distractor choices (pSD) calculated in a sliding 20-ms window, moving at 10 ms increments. Solid vertical lines are the median, 25th, and 75th percentiles of the full SRT distributions. In both monkeys, the proportion of salient distractor choices is significantly higher (indicated with *) for the 12.5% fastest responses (first octile, left of the dashed vertical line) than in the second through fourth octiles (chi-squared test, M1: Χ² (1) = 8.55, P < 0.01; M2: Χ² (1) = 21.41, P < 0.001).

Fig. 3.

V4 activity during visual search reveals the time course of pop-out and pop-in. (A) Neuronal responses in area V4 responses on correct trials. Average V4 activity elicited by the target (T, green trace), nonsalient distractors (ND, gray trace) and the salient distractor (SD, red trace) averaged across animals (Left) and for individual monkeys (M1: Middle panel; M2: Right panel). Shaded area corresponds to SEM across recording sites. Black arrows indicate the average reaction time (for M1 this was later than 250 ms and is not depicted). The light gray areas indicate the time window used for statistical testing of the response modulation, with * indicating P < 0.001 with a paired t test (green: T-ND; red: SD-ND). (B) Time course of neuronal target and salient distractor modulation. Top row, difference in activity elicited by the target and nonsalient distractor (T-ND; nonoverlapping 10 ms time bins) pooled across monkeys (Left) and individual animals (Middle and Right). Green bars indicate significant epochs at P < 0.05 (t test with Bonferroni correction for multiple comparisons). Bottom row, difference in activity elicited by the salient distractor and nonsalient distractor (SD-ND) with the red bars indicating P < 0.05 (t test, Bonferroni correction). In both animals, there is an initial epoch of salient distractor enhancement, followed by suppression, later than 150 ms. Colored arrows indicate the latency of target enhancement (green) and salient distractor suppression (red).

Fig. 4.

Pop-out and Pop-in. During the early phase of the V4 response (Middle) to a visual search stimulus (Left), both the shape and color singletons pop-out. In a later phase of the response (Right), top-down influences invert the pop-out of the salient color distractor into pop-in.