Visual search in scenes involves selective and nonselective pathways

Abstract

How does one find objects in scenes? For decades, visual search models have been built on experiments in which observers search for targets, presented among distractor items, isolated and randomly arranged on blank backgrounds. Are these models relevant to search in continuous scenes? This article argues that the mechanisms that govern artificial, laboratory search tasks do play a role in visual search in scenes. However, scene-based information is used to guide search in ways that had no place in earlier models. Search in scenes might be best explained by a dual-path model: a 'selective' path in which candidate objects must be individually selected for recognition and a 'nonselective' path in which information can be extracted from global and/or statistical information.

Figure 1

Find the four purple and green Ts. Even though it is easy to identify such targets, this task requires search.

Figure 2

Find the desaturated color dots. Colors are only an approximation of the colors that would be used in a carefully calibrated experiment. The empirical result is that it is much easier to find the pale red (pink) targets than to find pale green or blue.

Figure 3

Find the loaf of bread in each panel.

Figure 4

A two-pathway architecture for visual processing. A selective pathway can bind features and recognize objects, but it is severely capacity-limited. The limit is shown as a “bottleneck” in the pathway. Access to the bottleneck is controlled by guidance mechanisms that allow items that are more likely to be targets preferential access to feature binding and object recognition. Classic guidance, cartooned in the box above the bottleneck, gives preference to items with basic target features (e.g. color). This paper posits scene guidance (semantic and episodic), with semantic guidance derived from a non-selective pathway. This non-selective pathway can extract statistics from the entire scene, enabling a certain amount of semantic processing but not precise object recognition.

Figure 5

What do you see? And how does that change when you are asked to look for an untilted bird or trees with brown trunks and green boughs? It is proposed that a non-selective pathway would ‘see’ image statistics like average color or orientation in a region. It could get the ‘gist’ of forest and, perhaps, the presence of animals. It would not know which trees had brown trunks or which birds were tilted.