A direct comparison of sound and vibration as sources of stimulation for a sensory substitution glove

Abstract

Sensory substitution devices (SSDs) facilitate the detection of environmental information through enhancement of touch and/or hearing capabilities. Research has demonstrated that several tasks can be successfully completed using acoustic, vibrotactile, and multimodal devices. The suitability of a substituting modality is also mediated by the type of information required to perform the specific task. The present study tested the adequacy of touch and hearing in a grasping task by utilizing a sensory substitution glove. The substituting modalities inform, through increases in stimulation intensity, about the distance between the fingers and the objects. A psychophysical experiment of magnitude estimation was conducted. Forty blindfolded sighted participants discriminated equivalently the intensity of both vibrotactile and acoustic stimulation, although they experienced some difficulty with the more intense stimuli. Additionally, a grasping task involving cylindrical objects of varying diameters, distances and orientations was performed. Thirty blindfolded sighted participants were divided into vibration, sound, or multimodal groups. High performance was achieved (84% correct grasps) with equivalent success rate between groups. Movement variables showed more precision and confidence in the multimodal condition. Through a questionnaire, the multimodal group indicated their preference for using a multimodal SSD in daily life and identified vibration as their primary source of stimulation. These results demonstrate that there is an improvement in performance with specific-purpose SSDs, when the necessary information for a task is identified and coupled with the delivered stimulation. Furthermore, the results suggest that it is possible to achieve functional equivalence between substituting modalities when these previous steps are met.

Keywords: Grasping; Multimodal; Sensory substitution devices; Visual impairments.

Fig. 1

Sensation levels for each sensory modality as a function of the intensity steps. Note. Mean numerical estimates of sensation as a function of intensity level for vibration (in red) and sound (in blue) in the magnitude estimation task. The sixth step (black dot) was used as reference intensity and was given a value of 50 (arbitrary scale). The dashed lines represent the best fits to the data

Fig. 2

Picture of the sensory substitution glove. Note. Picture of the sensory substitution glove. The device consists of a regular polyester glove with six reflective markers (white semispheres) and two vibrotactile coin motors (one on the index finger and one on the thumb) which were inside a pocket

Fig. 3

Example of a successful trial. Note. Example of a successful trial using only acoustic stimulation is presented. In Panel A, the displacement of the index finger (red line), the thumb (blue line), and the average trajectory (black line), in the x–y coordinates. Panel B illustrates the temporal evolution of the distance between index finger and thumb during the trial. The onset of grasping and the maximum hand aperture were highlighted with red dots. Moreover, the onset of the was used to split the reaching from the grasping phase

Fig. 4

Summary of the survey. Note Percentage of affirmative answers to the survey questions by participants in the multimodal condition