Combination of conflicting visual and non-visual information for estimating actively performed body turns in virtual reality

Abstract

Whereas constant-weight linear models suffice for understanding many phenomena in the domain of perception and action, how the weights given to each sensory input are determined remains an open question. Notably, it has been suggested that weighting depends on the sensory context (e.g. the inconsistency between sensory signals) as well as on the subject. In the present study, the problem of non-linearity in multisensory interaction for estimating actively performed body turns was addressed at the level of group and individual data. Standing subjects viewed a virtual corridor in which forward movements were simulated at a constant linear velocity, and rotations were actually performed. Subjects were asked to learn the trajectory and then reproduce it from memory in total darkness. In the baseline condition, the relative amplitudes of visual and non-visual information for the performed rotations were the same, but were systematically manipulated in six 'sensory conflict' conditions. The subjects performed the task in these seven conditions 10 times (10 sessions), with a delay of at least 2 days between sessions. Five subjects placed more weight on visual than on non-visual information. The other 5 subjects placed more weight on non-visual than on visual information. Interestingly, the difference between 'visual' and 'non-visual' subjects in their use of conflicting information seemed to be accentuated by the fact of becoming aware of the sensory conflict. In all subjects, conflicting sensory inputs were combined in a linear way in order to estimate the angular displacements. However, signatures of non-linearity were detected when the data corresponding to the day on which subjects became aware of the conflict were considered in isolation. The present findings support the hypothesis that subjects used conflicting visual and non-visual information differently according to individual 'perceptive styles' (bottom-up processes) and that these 'perceptive styles' were made more observable by the subjects changing their perceptive strategy, i.e. re-weighting (top-down processes).