Perception of Upright: Multisensory Convergence and the Role of Temporo-Parietal Cortex

Abstract

We inherently maintain a stable perception of the world despite frequent changes in the head, eye, and body positions. Such "orientation constancy" is a prerequisite for coherent spatial perception and sensorimotor planning. As a multimodal sensory reference, perception of upright represents neural processes that subserve orientation constancy through integration of sensory information encoding the eye, head, and body positions. Although perception of upright is distinct from perception of body orientation, they share similar neural substrates within the cerebral cortical networks involved in perception of spatial orientation. These cortical networks, mainly within the temporo-parietal junction, are crucial for multisensory processing and integration that generate sensory reference frames for coherent perception of self-position and extrapersonal space transformations. In this review, we focus on these neural mechanisms and discuss (i) neurobehavioral aspects of orientation constancy, (ii) sensory models that address the neurophysiology underlying perception of upright, and (iii) the current evidence for the role of cerebral cortex in perception of upright and orientation constancy, including findings from the neurological disorders that affect cortical function.

Keywords: Bayesian; cerebral cortex; ocular torsion; orientation constancy; spatial orientation; subjective visual vertical; temporo-parietal cortex; upright perception.

Figure 1

Perception of upright and sensory reference frames: The head, eye, and the world reference frames are all aligned in upright position along the gravitational vertical (A), but when the head is tilted, the ocular counter-roll only partially compensates for the amount of head tilt (gain about 10–25%), which results in a separation of the sensory reference frames that encode head-in-space and eye/retina-in-head orientations (B). Despite these differences, visual perception remains in upright orientation (C). Therefore, the brain—like any other sensorimotor system—must be able to integrate sensory inputs into a common reference frame to maintain a coherent perception of upright.

Figure 2

Subjective visual vertical (SVV) measurement with the line stimulus (solid orange) presented at a random orientation in each trial (A). In the forced-choice paradigm, the task is to report whether the line is tilted to the right or to the left of the perceived upright orientation (dashed orange) (B). SVV is then determined by fitting a psychometric curve to the responses from all trials and is calculated as the value on the curve at which the probability of left or right responses is 50% (point of subjective equality). In the active-adjustment paradigm, the line stimulus (solid orange) is adjusted (direction shown by arrow) to the perceived upright orientation (dashed orange) (C). In this paradigm, SVV is calculated as the average value from all trials. The true vertical is shown by the dashed white line (B,C).

Figure 3

Systematic errors of subjective visual vertical (SVV): healthy individuals typically have SVV errors within 2° of earth vertical in upright position. At large tilt angles (usually greater than 60°), SVV errors are deviated toward the tilt direction, which reflect “underestimation” of upright orientation (known as the Aubert or A-effect). At smaller tilt angles (usually less than 60°), however, SVV errors are often opposite to the tilt direction, which reflect “overestimation” of upright orientation (known as the Entgegengesetzt or E-effect).

Figure 4

Schematic presentation of Mittelstaedt’s idiotropic vector model: visual line orientation on the retina and head tilt are the two sensory inputs in this model. The visual signal is accurate, but the head tilt signal $(\widehat{\rho })$ shows increasing errors with head tilt (a range of ±90° tilt is shown in black graphs). As part of the central neural processing, vectorial summation of the head tilt signal $(\widehat{\rho })$ and the head-fixed idiotropic vector (M_Z) yields the compensatory tilt signal (β). The compensatory tilt signal and the visual signal are then added to obtain an internal estimate of the upright orientation [i.e., subjective visual vertical (SVV)].

Figure 5

Schematic presentation of Bayesian spatial perception model: various sensory modalities are integrated into a common spatial reference frame to determine upright orientation. A vertical line (line in space) is presented in front of a tilted observer (head in space) (a range of ±90° tilt is shown in black graphs). Signal ${\widehat{H}}_S$, encoding head orientation in space, is accurate but contaminated by Gaussian noise. Head tilt results in ocular counter-roll (OCR) and signal ${\widehat{E}}_H$, encoding eye-in-head orientation, is also contaminated by independent noise. As part of central neural processing, the estimates of head-in-space $\left({\tilde{H}}_S\right)$ and eye-in-head $\left({\tilde{E}}_H\right)$ are generated separately from the likelihoods and priors of head tilt and torsional eye position (i.e., ocular torsion). These estimates are integrated to generate eye-in-space estimate $\left({\tilde{E}}_S\right)$, which is then integrated with retinal signal (line on retina) to obtain an internal estimate of the upright orientation [i.e., subjective visual vertical (SVV)].

Figure 6

Schematic representation of the sensory integration model: body sensors, neck sensors, and otoliths provide information about the body in-space, head-on-body, and head-in-space positions, respectively. As part of the central neural processing, the neck and body signals undergo coordinate transformation to indirectly encode head-in-space orientation. Overall, the optimal head-in-space estimate $\left({\tilde{H}}_S\right)$ is obtained by the relative weights of the otolith information (W_HD, blue pathway), coordinate-transformed information from the body and neck sensors (W_HI, green pathway), and the head prior information (W_HP, gray pathway). The head-in-space estimate $\left({\tilde{H}}_S\right)$ is then integrated with eye-in-head estimate $\left({\tilde{E}}_H\right)$ and line orientation on the retina to obtain an internal estimate of the upright orientation [i.e., subjective visual vertical (SVV)].

Figure 7

Schematic representation of the periodic subjective visual vertical (SVV) modulation by the frame orientation (solid line) during head tilt: frame tilt orientations close to the subject’s upright perception (dashed line) usually result in an “attractor bias” (i.e., toward the direction of the frame tilt), while there is a “detractor bias” at angles beyond 45° and up to 90° (i.e., away from the direction of the frame tilt). These biases caused by the frame orientation can either attenuate or accentuate SVV errors, depending on head tilt position (e.g., here 20° head tilt to the left).

Figure 8

Subjective visual vertical (SVV) and torsional eye position measured simultaneously before, during, and after prolonged head tilts (~15 min) in 12 subjects (15): data points represent SVV or ocular torsion from 100 trials during 20° head tilts to the right and left. Error bars correspond with SEM across subjects. The SVV drift is in the same direction as the head tilt, and when the head returns to upright position there is an aftereffect, also in the same direction as the head tilt. Changes in ocular torsion do not correspond to the SVV drift or aftereffect.

Figure 9

Approximate projections of the cortical areas associated with subjective visual vertical deviation based on anatomical locations of the average lesion areas from seven studies. The color map shows the degree of overlap among cortical involvement in these studies with maximum convergence around the temporo-parietal junction. The average age of the patients in years and the average time from the stroke in days are included for each study.

Figure 10

Simultaneous subjective visual vertical (SVV) and ocular torsion recordings during prolonged left head tilt of 20° in a single subject (500 trials ~15 min) [data from Ref. (260)]: SVV shift from transcranial magnetic stimulation (TMS) at SMGp (red) is shown along with the SVV shift from the sham stimulation (i.e., no TMS) (blue) (A). In both traces, there is a gradual drift over time toward the left (i.e., in the same direction as the head tilt), but the SVV shift from TMS is larger with a deviation opposite to the direction of the head tilt. Ocular torsion shift from TMS at SMGp (red) is not different form the sham stimulation (blue) (B). As opposed to SMGp, SVV shift from TMS at another cortical location outside of TPJ (orange) is smaller than the sham stimulation with a deviation in the same direction as the head tilt (C). PMC, primary motor cortex; SMG, supramarginal gyrus; AG, angular gyrus; STG, superior temporal gyrus; SF, Sylvian fissure.

Figure 11

Schematic showing the directions of subjective visual vertical (SVV) tilt and postural deviation in pusher syndrome. In these patients, SVV and postural vertical perception deviate away from the side of the lesion (X), matching the direction of postural tilt (i.e., lateropulsion) as well as the pushing behavior toward the paretic side. Therefore, patients with pushing behavior seem to actively align their body with erroneous upright and postural estimates.