Multisensory space: from eye-movements to self-motion

Abstract

We perceive the world around us as stable. This is remarkable given that our body parts as well as we ourselves are constantly in motion. Humans and other primates move their eyes more often than their hearts beat. Such eye movements lead to coherent motion of the images of the outside world across the retina. Furthermore, during everyday life, we constantly approach targets, avoid obstacles or otherwise move in space. These movements induce motion across different sensory receptor epithels: optical flow across the retina, tactile flow across the body surface and even auditory flow as detected from the two ears. It is generally assumed that motion signals as induced by one's own movement have to be identified and differentiated from the real motion in the outside world. In a number of experimental studies we and others have functionally characterized the primate posterior parietal cortex (PPC) and its role in multisensory encoding of spatial and motion information. Extracellular recordings in the macaque monkey showed that during steady fixation the visual, auditory and tactile spatial representations in the ventral intraparietal area (VIP) are congruent. This finding was of major importance given that a functional MRI (fMRI) study determined the functional equivalent of macaque area VIP in humans. Further recordings in other areas of the dorsal stream of the visual cortical system of the macaque pointed towards the neural basis of perceptual phenomena (heading detection during eye movements, saccadic suppression, mislocalization of visual stimuli during eye movements) as determined in psychophysical studies in humans.

Figure 1. Location of the ventral intraparietal area (area VIP) in the depth of the intraparietal sulcus

Borders between grey and white matter are based on histological sections. Neighbouring areas 7a, LIP (lateral intraparietal area) and 5 are indicated.

Figure 2. Comparison of visual responsiveness

In areas MST and VIP, the median visual response during fixation long before a saccade was normalized to a value of 100%. The response was computed for a pre-selected response window (75–125 ms after stimulus onset). Responses in motion-sensitive areas MST and VIP were statistically different in the three stimulus periods (P < 0.001, repeated measures ANOVA on ranks). A pair-wise multiple comparison (Tukey test) revealed that the Peri-responses were significantly smaller than the responses in the Pre- and the Post-condition. In addition, in area VIP, the responses in the Post-condition were significantly stronger than in the Pre-condition (P < 0.05, Tukey test).

Figure 3. Comparison of neural and behavioural data

For the behavioural data, the horizontal axis shows time between stimulus presentation and saccade onset, the right vertical axis indicates normalized contrast sensitivity as taken from Diamond et al. (2000). Neuronal data were shifted along the time axis in order to correct for response and processing latencies and represent neuronal excitability (left vertical axis) of the MST population (blue curve) and the VIP population (red curve). The time course of neuronal excitability in both motion areas of the macaque shows a good qualitative match with the time course of perceptual loss of sensitivity around saccades in human subjects (modified from Bremmer et al. 2009).

Figure 4. Activation pattern related to OKN relative to baseline as a result of the group analysis

All areas were significant at P < 0.05 corrected. R, right hemisphere; L, left hemisphere; The z-value indicates the coordinate of the horizontal image, VIP, ventral intraparietal area.

Figure 5. Retinal flow fields and related neuronal responses

The left columns show the retinal flow fields seen by an observer moving in a rightward heading direction on top of a ground plane. All three panels display the same heading (arrow) but differ in terms of simulated eye movements. The pattern as shown in the top panel occurs in the absence of eye movements (Gain = 0.0). In the middle and bottom panel, eye movements which spontaneously occur in primates are included (Gain = 0.5 (middle) and Gain = 1.0 (bottom)). These eye movements track the motion in the direction of gaze (circle) and distort the structure of the flow on the retina, generating a motion pattern that resembles a spiral and in which the motion in gaze direction (circle) is minimized. Invariant responses to heading should be the same in all three cases, since heading is identical although flow structure is different. The right column shows responses from a neuron responding to rightward heading irrespective of eye movements. Histograms show the neuron's firing rate over time during presentation of optic flow stimuli.

Figure 6. Action congruency of responses to visual, tactile and vestibular stimulation

Shading and arrows on the animal drawings indicate, respectively, the extent of the somatosensory receptive field and the directional preference of the tactile stimulus. Insets on top of the drawings indicate the cell identification, whether the neuron was tested for vestibular responses, the optic flow response and the directional selectivity vector.

Figure 7. Spatially congruent visual and auditory RFs of an individual VIP neuron

Neuronal responses are colour coded corresponding to high discharge values. The data have been recorded while the monkey fixated a central target. The two RFs largely overlapped, and the hotspots were almost identical.