Perceptual learning of view-independence in visuo-haptic object representations

Abstract

We previously showed that cross-modal recognition of unfamiliar objects is view-independent, in contrast to view-dependence within-modally, in both vision and haptics. Does the view-independent, bisensory representation underlying cross-modal recognition arise from integration of unisensory, view-dependent representations or intermediate, unisensory but view-independent representations? Two psychophysical experiments sought to distinguish between these alternative models. In both experiments, participants began from baseline, within-modal, view-dependence for object recognition in both vision and haptics. The first experiment induced within-modal view-independence by perceptual learning, which was completely and symmetrically transferred cross-modally: visual view-independence acquired through visual learning also resulted in haptic view-independence and vice versa. In the second experiment, both visual and haptic view-dependence were transformed to view-independence by either haptic-visual or visual-haptic cross-modal learning. We conclude that cross-modal view-independence fits with a model in which unisensory view-dependent representations are directly integrated into a bisensory, view-independent representation, rather than via intermediate, unisensory, view-independent representations.

Figure 1

Two models of bisensory view-independence in which this is either (a) derived directly from unisensory view-dependent representations or (b) gated by separate unisensory view-independent representations.

Figure 2

Example object shown (left) unrotated and (right) rotated 180° about the y-axis.

Figure 3

A schematic timeline for the experiments: after recording within-modal baseline performance, participants performed 10 learning trials. In Experiment 1 these were either within-modal visual or haptic and in Experiment 2, either visual study-haptic test or vice-versa. The effect of learning was assessed with final within-modal visual and haptic trials.

Figure 4

Mean visual and haptic within-modal recognition accuracy for unrotated and rotated objects before and after (a) within-modal haptic learning and (b) within-modal visual learning. Error bars = s.e.m.

Figure 5

Mean visual and haptic within-modal recognition accuracy for unrotated and rotated objects before and after (a) cross-modal visual-haptic learning and (b) cross-modal haptic-visual learning. Error bars = s.e.m.