Does the brain's ventral visual pathway compute object shape?

Abstract

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Keywords: deep neural networks; dorsal stream; object recognition; shape perception; ventral stream; viewpoint-invariance.

Figure 1.. Object images with intact and disrupted global shapes.

(A) Examples of airplanes where the spatial arrangement of features is preserved. Despite radically different local features, a common spatial arrangement, or structure, elicits the same percept of shape. (B) Examples of airplanes where the spatial arrangement of features is disrupted or partially occluded via foreshortening.

Figure 2.. Slight image perturbations change the firing rate of individual ventral neurons.

(A) Example of a clean preferred (red) and non-preferred (blue) image categories for an individual neuron, as well as non-preferred images with various degrees of perturbation as specified by ε. (B) Raster plot illustrating the firing rate of the neuron in response to its preferred category (red), as well as to a non-preferred category (blue) following different degrees of perturbation. (C) The normalized (norm.) firing rate to preferred images (dashed red line) and non-preferred images (solid blue line) following different degrees of perturbation. By ε = 10, the firing to the non-preferred image category exceeds the preferred image category. Figure adapted, with permission, from [58].

Figure 3.. Greater sensitivity to local features than global shape in the ventral pathway.

(A) Example stimuli and results from [60] as measured behaviorally from human observers (top), the human ventral visual pathway (middle), and deep neural networks (DNNs, bottom). Each image triplet depicts the multidimensional scaling (MDS) of image similarities. The top two images of each triplet depict synthetic images where the arrangement of features has been scrambled, and the bottom image of each triplet depicts the original image where the arrangement of features remains intact. The distances between each image in a triplet reflect their similarities. (Top) Human observers readily grouped feature scrambled images together and discriminated them from intact images. By contrast, the multivariate responses of the (middle) ventral pathway and (bottom) DNNs showed no such grouping, with equal distances between each image. (B) Example stimuli and results from [61]. (Top) Texform examples are shown alongside their real-world counterparts for animate/inanimate and large/small categories. (Bottom) Ventral pathway activation maps for the original images and their texform counterparts. Although unrecognizable by human observers, texforms elicit the same large-scale topographic organization along the dimensions of object animacy and size as real-world objects.

Figure 4.. An expanded brain network for object recognition.

In this schematic depiction of the visual system, the ventral pathway (V1 to ATL) acts much like a DNN (bottom) – extracting increasingly complex local object features, but not a complete shape. Instead, structural information describing the global shape of an object, but not its individual features (top; depicted as a red skeleton), may be computed in dorsal visual pathway regions such as IPS. This information is then sent to the ventral pathway to form a complete object representation. Abbreviations: ATL, anterior temporal lobe; DNN, deep neural network; IPS, intraparietal sulcus; L1–L6, layers 1–6; LOC, lateral occipital complex; V1–V4, visual areas 1–4.