A Bayesian Account of Visual-Vestibular Interactions in the Rod-and-Frame Task

Abstract

Panoramic visual cues, as generated by the objects in the environment, provide the brain with important information about gravity direction. To derive an optimal, i.e., Bayesian, estimate of gravity direction, the brain must combine panoramic information with gravity information detected by the vestibular system. Here, we examined the individual sensory contributions to this estimate psychometrically. We asked human subjects to judge the orientation (clockwise or counterclockwise relative to gravity) of a briefly flashed luminous rod, presented within an oriented square frame (rod-in-frame). Vestibular contributions were manipulated by tilting the subject's head, whereas visual contributions were manipulated by changing the viewing distance of the rod and frame. Results show a cyclical modulation of the frame-induced bias in perceived verticality across a 90° range of frame orientations. The magnitude of this bias decreased significantly with larger viewing distance, as if visual reliability was reduced. Biases increased significantly when the head was tilted, as if vestibular reliability was reduced. A Bayesian optimal integration model, with distinct vertical and horizontal panoramic weights, a gain factor to allow for visual reliability changes, and ocular counterroll in response to head tilt, provided a good fit to the data. We conclude that subjects flexibly weigh visual panoramic and vestibular information based on their orientation-dependent reliability, resulting in the observed verticality biases and the associated response variabilities.

Keywords: Bayesian inference; internal models; multisensory integration; rod-and-frame task; spatial orientation; verticality perception.

Figure 1.

Experimental procedure of the rod-and-frame task. After presenting a square frame for 250 ms, a rod is briefly (33 ms) flashed within the frame. When the rod disappears, the square remains visible until the subjects responds whether the rod was rotated CW or CCW from upright. A 500-ms black screen is presented before the start of a new trial.

Figure 2.

Schematic representation of the refined Bayesian optimal integration model for visual context. Physical signals about the retinal frame orientation (${\theta }_R$), true head-in-space orientation (H_Sact), and prior knowledge about likely head orientations (H_P) are transformed into sensory signals, denoted by the hat symbol (ˆ). Sensory signals are assumed to be accurate but contaminated with Gaussian noise (κ, σ_HS, and σ_HP, respectively). For an optimal estimate of head-in-space orientation, denoted by a tilde (˜), the model integrates the contextual likelihood $P\left({\widehat{\theta }}_R|H_S\right)$ together with the vestibular likelihood $P\left({\widehat{H}}_S|H_S\right)$ and the head-in-space prior $P\left(H_S\right)$ This translates into multiplying the individual probability distributions: $P\left({\tilde{H}}_S|{\widehat{H}}_S,{\widehat{\theta }}_R\right)=P\left({\widehat{H}}_S|H_S\right)\ast P\left({\widehat{\theta }}_R|H_S\right)\ast P\left(H_S\right)$ The maximum of the resulting posterior distribution (MAP) is selected as the perceived head-in-space orientation (${\tilde{H}}_S$), whereas the width of the curve is a measure of the response variability. The perceived orientation of the line in space is then obtained by a coordinate transformation using the eye-in-head orientation (${\tilde{E}}_H$, uncompensated ocular counterroll) and the retinal rod orientation estimate (${\tilde{L}}_E$ , assumed to be veridical). The probability distributions in the figure represent the case in which the subject is seated upright (H_S = 0°) with the frame displayed upright (θ_R = 0°).

Figure 3.

Probability of CW responses plotted against rod orientation for three example frame orientations (20° CCW, upright, and 20°CW) in a representative subject (black circles). Red solid lines represent the psychometric fits that quantify the bias (µ, dashed line) and variability (σ, inversely related to the slope) of the subject in each panel.

Figure 4.

Bias and response variability plotted against frame orientation in a representative subject (circles) for all conditions. The biases and variabilities obtained from Figure 3 are highlighted in blue. Red solid lines represent the best fit of the Bayesian optimal integration model (Table 2). The dashed lines in the vestibular bias plot indicate the dark subjective visual vertical task (SVV).

Figure 5.

Mean bias and variability plots across all subjects for all conditions. Error bars represent the SD across subjects. The red solid lines on top of the data are the mean of the best fit across all subjects, with the shaded areas representing the SE on the model fit.

Figure 6.

Prior (red), otoliths (blue), and visual contextual (green) weight distributions plotted against frame orientation in the different conditions. Shaded areas represent the SE across all subjects.

Figure 7.

Bias plotted against frame orientation in the baseline and visual condition for subject S5. Red solid lines indicate the best fit of the Bayesian optimal integration model. R ² values indicate the goodness of fit of the model to the data.

Figure 8.

Simulations of the Bayesian optimal integration model for bias and response variability in patients with complete vestibular function loss. Shaded areas represent the SE on the model simulations.