Studying the visual brain in its natural rhythm

Abstract

How the brain fluidly orchestrates visual behavior is a central question in cognitive neuroscience. Researchers studying neural responses in humans and nonhuman primates have mapped out visual response profiles and cognitive modulation in a large number of brain areas, most often using pared down stimuli and highly controlled behavioral paradigms. The historical emphasis on reductionism has placed most studies at one pole of an inherent trade-off between strictly controlled experimental variables and open designs that monitor the brain during its natural modes of operation. This bias toward simplified experiments has strongly shaped the field of visual neuroscience, with little guarantee that the principles and concepts established within that framework will apply more generally. In recent years, a growing number of studies have begun to relax strict experimental control with the aim of understanding how the brain responds under more naturalistic conditions. In this article, we survey research that has explicitly embraced the complexity and rhythm of natural vision. We focus on those studies most pertinent to understanding high-level visual specializations in brains of humans and nonhuman primates. We conclude that representationalist concepts borne from conventional visual experiments fall short in their ability to capture the real-life visual operations undertaken by the brain. More naturalistic approaches, though fraught with experimental and analytic challenges, provide fertile ground for neuroscientists seeking new inroads to investigate how the brain supports core aspects of our daily visual experience.

Keywords: Bodies; Face patches; Faces; Free viewing; IT; Inferior temporal cortex; Naturalisitic; Neural representation; Perception; Primate; Vision.

Figure 1.

Fixed-gaze paradigms used in humans and nonhuman primates. The dominance of these paradigms, where the subject is immobilized, the head restrained, and the gaze maintained on a small point, has strongly shaped theories of visual neuroscience.

Figure 2.

Responses of single neurons in the macaque anterior fundus (AF) face patch to flashed images from different categories (A) and to a five-minute video depicting primate social interaction (B). A. Each row in the heat map corresponds to an individual neuron, and each column is the average spiking response to a particular image. The image categories are shown at the bottom, revealing a strong bias for responding to faces, as is typical among neurons in macaque face patches. On the right are the mean category responses of eight example neurons. B. Movie responses are shown for the same eight example neurons. In each panel, the action potentials for multiple presentations of the movie are shown in the faint raster plots, revealing a strong repeatability across trials. The spike density function is shown in the blue lines superimposed on the rasters. A striking feature of the movie response time courses is that they are largely uncorrelated among these nearby neurons, despite their similar categorical responses shown in A.

Figure 3.

Data-driven approaches to understand the functional specialization of single neurons in high-level visual cortex. A. Single-unit mapping of individual time courses of neurons onto voxels across the brain (Park et al., 2017). This method uses fMRI-responses to video content as a way to interpret the response time courses of single neurons. Within macaque face patches, neighboring single neurons show highly divergent correlational maps with the rest of the brain, indicating a heterogeneity of long-range inputs and visual operations in the local neural population. B. Single-unit based design of visual images using deep learning methods (Ponce et al., 2019). Optimized stimuli are determined through multiple iterations of neural recording and stimulus modification. Examples of final optimized patterns are shown (dashed lines are for illustration purposes and do not indicate recording sites in the temporal cortex).

Figure 4.

Real world geometrical considerations that come to the fore when visual experiments involve unrestrained animals. A. Constraints of real object size on the brain’s interpretation of visual images. The practice of describing the size of visual objects in degrees of visual angle, which corresponds to the retinal geometry, is inherently ambiguous and may not match the brain’s encoding of objects. High-level visual specializations may instead be more directly related to objects’ physical sizes and positions within the scene, including the distance to the observer. B. Deducing 3D volumetric structure and spatial relations through self-motion. The important role of self-induced movement in understanding the three-dimensional structure of a scene is generally omitted from visual experiments. However, it may be the most fundamental cue by which animals understand both near and far three-dimensional structure, and it may be deeply embedded within cortical circuits concerned with visual perception and action.