Sensory substitution: The affordance of passability, body-scaled perception, and exploratory movements

Abstract

The theory of affordances states that perception is of environmental properties that are relevant to action-capabilities of perceivers. The present study illustrates how concepts and methodological tools from the theory of affordances may help to advance research in the field of sensory substitution. The sensory substitution device (SSD) that was used consisted of two horizontal rows of 12 coin motors that each vibrated as a function of the distance to the nearest object. Sixty blindfolded participants used the SSD to explore virtual horizontal apertures with different widths. They were asked to judge the passability of the apertures. Participants with narrow shoulders judged narrower apertures as passable than participants with wide shoulders. This difference disappeared when aperture width was scaled to shoulder width, demonstrating that perception was body scaled. The actual aperture width was closely related to aspects of the exploratory movements and to aspects of the vibrotactile stimulation that was obtained with the exploratory movements. This implies that the exploratory movements themselves and the vibrotactile stimulation were both informative about the aperture width, and hence that the perception of passability may have been based on either of them or on a global variable that spans vibrotactile as well as kinaesthetic stimulation. Similar performance was observed for participants who accomplished the 7-trial familiarization phase with or without vision, meaning that practice with vision is not indispensable to learn to use the SSD.

Fig 1. Experimental material.

A. Schematic drawing of experimental setup. B. From upper left to lower right corner: wooden rod, blindfold, reflective markers, elastic band with motors, and backpack with microcontrollers and battery.

Fig 2. Variables used to compute exploratory movement variables.

x = horizontal position; y = depth position; and h = heading direction.

Fig 3. Logistic fits on proportions of affirmative judgments.

A. Fitted proportions for both shoulder-width groups as a function of aperture width. B. Fitted proportions for both groups as a function of π-number (aperture width / shoulder width). The edges of the colored regions indicate the logistic fits ± one standard deviation.

Fig 4. Exploratory movements in a representative trial.

A. Starting point, allowed limit in y position, and participant’s translation. B. Time-series of y position plotted as a function of heading angle. C. Heading angle as a function of time.

Fig 5. Movement variables as a function of aperture width.

A. Linear displacement (for visual and haptic training groups). B. Angular displacement. C. Minimum distance. D. Average number of active motors. Error bars represent standard errors of the mean (SEM).

Fig 6. Aperture-normalized exploratory variables as a function of type of error.

A. Linear displacement. B. Angular displacement. C. Minimum distance. D. Average number of active motors. Error bars represent standard errors of the mean (SEM).