Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2021 Sep 30;21(19):6575.
doi: 10.3390/s21196575.

Audio-Tactile Rendering: A Review on Technology and Methods to Convey Musical Information through the Sense of Touch

Byron Remache-Vinueza  1   2 , Andrés Trujillo-León  1   3 , Mireya Zapata  4 , Fabián Sarmiento-Ortiz  2 , Fernando Vidal-Verdú  1   3
Affiliations
Review

Audio-Tactile Rendering: A Review on Technology and Methods to Convey Musical Information through the Sense of Touch

Byron Remache-Vinueza et al. Sensors (Basel). .

Abstract

Tactile rendering has been implemented in digital musical instruments (DMIs) to offer the musician haptic feedback that enhances his/her music playing experience. Recently, this implementation has expanded to the development of sensory substitution systems known as haptic music players (HMPs) to give the opportunity of experiencing music through touch to the hearing impaired. These devices may also be conceived as vibrotactile music players to enrich music listening activities. In this review, technology and methods to render musical information by means of vibrotactile stimuli are systematically studied. The methodology used to find out relevant literature is first outlined, and a preliminary classification of musical haptics is proposed. A comparison between different technologies and methods for vibrotactile rendering is performed to later organize the information according to the type of HMP. Limitations and advantages are highlighted to find out opportunities for future research. Likewise, methods for music audio-tactile rendering (ATR) are analyzed and, finally, strategies to compose for the sense of touch are summarized. This review is intended for researchers in the fields of haptics, assistive technologies, music, psychology, and human-computer interaction as well as artists that may make use of it as a reference to develop upcoming research on HMPs and ATR.

Keywords: haptic music player; human computer interaction; musical haptic wearables; musical haptics; sensory substitution systems; tactile rendering; vibrotactile feedback; vibrotactile music composition.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Number of HMP publications since 1992 until 2020.
Figure 2
Figure 2
Preliminary classification of musical haptics.
Figure 3
Figure 3
Examples of haptic player installations (HMPIs): (a) Deaf user experiencing music in a haptic chair, retrieved from in [8]. (b) Multimodal platform, retrieved from in [49]. (c) Desktop haptic knob installation, retrieved from in [50]. (d) Desktop haptic button installation, retrieved from in [51].
Figure 4
Figure 4
Examples of haptic music player-wearable devices (HMP-WDs): (a) Deaf user experiencing music with a haptic bracelet MuSS-Bits, retrieved from in [19]. (b) Haptic glove, retrieved from in [61]. (c) Haptic belt Hedonic Haptic Player, retrieved from in [35]. (d) Haptic jacket Body:Suit:Score, retrieved from in [65].
Figure 5
Figure 5
Spatialization of frequencies in a haptic chair, retrieved from in [32].
Figure 6
Figure 6
The Auris system for tactile rendering of melodic content, retrieved from in [40].
Figure 7
Figure 7
Different instrument tracks for TM implementation in the Model Human Cochlea, retrieved from in [14].
Figure 8
Figure 8
The Vibrochord, a vibrotactile musical input device VMID, retrieved from [66].
Figure 9
Figure 9
Example of rhythmic VMC created for the Hedonic Haptic Player, retrieved from in [35].
See this image and copyright information in PMC

References

    1. Schneider A., Feussner H. Biomedical Engineering in Gastrointestinal Surgery. Academic Press; Cambridge, MA, USA: 2017. Mechatronic Support Systems and Robots; pp. 387–441. - DOI
    1. Birnbaum D.M., Wanderley M.M. A Systematic Approach to Musical Vibrotactile Feedback; Proceedings of the International Computer Music Conference; Copenhagen, Denmark. 27–31 August 2007; pp. 397–404.
    1. Giordano M., Wanderley M.M. Perceptual and Technological Issues in the Design of Vibrotactile-Augmented Interfaces for Music Technology and Media; Proceedings of the International Workshop on Haptic and Audio Interaction Design; Daejeon, Korea. 18–19 April 2013; pp. 89–98.
    1. Verrillo R.T. Vibration Sensation in Humans. Music Percept. 1992;9:281–302. doi: 10.2307/40285553. - DOI
    1. Chafe C. Tactile Audio Feedback; Proceedings of the 19th International Computer Music Conference (ICMC 1993); Tokyo, Japan. 10–15 September 1993; pp. 76–79.