Efficient visual recalibration from either visual or haptic feedback: the importance of being wrong

Abstract

The human visual system adapts to the changing statistics of its environment. For example, the light-from-above prior, an assumption that aids the interpretation of ambiguous shading information, can be modified by haptic (touch) feedback. Here we investigate the mechanisms that drive this adaptive learning. In particular, we ask whether visual information can be as effective as haptics in driving visual recalibration and whether increased information (feedback from multiple modalities) induces faster learning. During several hours' training, feedback encouraged observers to modify their existing light-from-above assumption. Feedback was one of the following: (1) haptic only, (2) haptic and stereoscopic (providing binocular shape information), or (3) stereoscopic only. Haptic-only feedback resulted in substantial learning; the perceived shape of shaded objects was modified in accordance with observers' new light priors. However, the addition of continuous visual feedback (condition 2) substantially reduced learning. When visual-only feedback was provided intermittently (condition 3), mimicking the time course of the haptic feedback of conditions 1 and 2, substantial learning returned. The intermittent nature of conflict information, or feedback, appears critical for learning. It causes an initial, erroneous percept to be corrected. Contrary to previous proposals, we found no particular advantage for cross-modal feedback. Instead, we suggest that an "oops" factor drives efficient learning; recalibration is prioritized when a mismatch exists between sequential representations of an object property. This "oops" factor appears important both across and within sensory modalities, suggesting a general principle for perceptual learning and recalibration.

Figure 1.

Stimuli and methods. A, The visual–haptic setup and an example baseline trial. The upper-right and lower-left objects in this scene are generally perceived as convex and the others as concave, consistent with overhead lighting. The red asterisk signaled which object should be judged (via buttons labeled “in” and “out”). B, Left plot, Proportion convex judgments as a function of stimulus orientation for one observer: baseline data (red asterisks) and model fit (red line). Right plot, Orientations trained as convex (shaded gray region) and data from haptic trials (black asterisks). C, Schematic of the available shape information (monocular, binocular and haptic) at each stage of a training trial, for each condition.

Figure 2.

A, Light priors for individual observers for the Haptic condition (left) and Haptic&Stereo condition (right). Circles represent test blocks, while black squares give shape responses on training trials. Error bars show 95% confidence intervals from bootstrapping. Dashed horizontal lines give the observer's baseline light prior (red/green) and the trained light prior direction (black). Shaded gray strips indicate train–test sessions. B, C, Group results for the two feedback conditions during the 2 day training period (B) and after training (C). Asterisks indicate test blocks that differ significantly from baseline (p < 0.05). Error bars give ±1 SE across observers.

Figure 3.

A, The progression of cue availability and the resultant percepts within training trials on a conflict trial where the stimulus is initially perceived as concave, but feedback indicates that it is convex (upper panel: Haptic condition, lower panel: Haptic&Stereo condition). B, Data from all three training conditions. Asterisks indicate test blocks where significant learning was observed in the Stereo feedback condition.