Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019;37(2):155-166.
doi: 10.3233/RNN-190898.

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

Katarzyna Cieśla  1   2 , Tomasz Wolak  1 , Artur Lorens  1 , Benedetta Heimler  2 , Henryk Skarżyński  1 , Amir Amedi  2   3
Affiliations

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

Katarzyna Cieśla et al. Restor Neurol Neurosci. 2019.

Abstract

Background: Hearing loss is becoming a real social and health problem. Its prevalence in the elderly is an epidemic. The risk of developing hearing loss is also growing among younger people. If left untreated, hearing loss can perpetuate development of neurodegenerative diseases, including dementia. Despite recent advancements in hearing aid (HA) and cochlear implant (CI) technologies, hearing impaired users still encounter significant practical and social challenges, with or without aids. In particular, they all struggle with understanding speech in challenging acoustic environments, especially in presence of a competing speaker.

Objectives: In the current proof-of-concept study we tested whether multisensory stimulation, pairing audition and a minimal-size touch device would improve intelligibility of speech in noise.

Methods: To this aim we developed an audio-to-tactile sensory substitution device (SSD) transforming low-frequency speech signals into tactile vibrations delivered on two finger tips. Based on the inverse effectiveness law, i.e., multisensory enhancement is strongest when signal-to-noise ratio is lowest between senses, we embedded non-native language stimuli in speech-like noise and paired it with a low-frequency input conveyed through touch.

Results: We found immediate and robust improvement in speech recognition (i.e. in the Signal-To-Noise-ratio) in the multisensory condition without any training, at a group level as well as in every participant. The reported improvement at the group-level of 6 dB was indeed major considering that an increase of 10 dB represents a doubling of the perceived loudness.

Conclusions: These results are especially relevant when compared to previous SSD studies showing effects in behavior only after a demanding cognitive training. We discuss the implications of our results for development of SSDs and of specific rehabilitation programs for the hearing impaired either using or not using HAs or CIs. We also discuss the potential application of such a set-up for sense augmentation, such as when learning a new language.

Keywords: Speech understanding in noise; cochlear implants; hearing impairment; multisensory rehabilitation; multisensory training; sensory substitution device; vibrotactile stimulation.

PubMed Disclaimer

Figures

Fig.1
Fig.1
A) vibrating interface of the Vibrating Auditory Stimulator; B) Matlab GUI for stimuli presentation and control; C) Speech Reception Threshold values obtained for auditory and auditory-tactile speech in noise stimulation, at the group level and in individual subjects [subject 6 showed an improvement from 0.3 to –3.0 SRT(dB)].
See this image and copyright information in PMC

References

    1. Abboud S., Hanassy S., Levy-Tzedek S., Maidenbaum S., Amedi A. (2014). EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution. Restorative Neurology and Neuroscience, 32(2), 247–257. - PubMed
    1. Agrawal Y., Platz E.A., Niparko J.K. (2008). Prevalence of hearing loss and differences by demographic characteristics among US adults: data from the National Health and Nutrition Examination Survey, 1999-2004. Archives of Internal Medicine, 168(14), 1522–1530. - PubMed
    1. Amedi A., Stern W., Camprodon J.A., Bermpohl F., Merabet L., Rotman S., Hemond C.C., Meijer P., Pascual-Leone A. (2007). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience, 10, 687–689. - PubMed
    1. Amieva H., Ouvrard C., Giulioli C., Meillon C., Rullier L., Dartigues J.F. (2015). Self-Reported Hearing Loss, Hearing Aids, and Cognitive Decline in Elderly Adults: A 25-Year Study. Journal of the American Geriatrics Society, 63(10), 2099–2104. - PubMed
    1. Anderson S., White-Schwoch T., Parbery-Clark A., Kraus N. (2013). Reversal of age-related neural timing delays with training. Proceedings of the National Academy of Sciences U S A, 4357–4362. - PMC - PubMed

Publication types