Drawing sounds: representing tones and chords spatially

Abstract

Research on the crossmodal correspondences has revealed that seemingly unrelated perceptual information can be matched across the senses in a manner that is consistent across individuals. An interesting extension of this line of research is to study how sensory information biases action. In the present study, we investigated whether different sounds (i.e. tones and piano chords) would bias participants' hand movements in a free movement task. Right-handed participants were instructed to move a computer mouse in order to represent three tones and two chords. They also had to rate each sound in terms of three visual analogue scales (slow-fast, unpleasant-pleasant, and weak-strong). The results demonstrate that tones and chords influence hand movements, with higher-(lower-)pitched sounds giving rise to a significant bias towards upper (lower) locations in space. These results are discussed in terms of the literature on forward models, embodied cognition, crossmodal correspondences, and mental imagery. Potential applications sports and rehabilitation are discussed briefly.

Keywords: Crossmodal correspondences; Embodied cognition; Mouse-tracking; Movement; Sound; Space; Valence.