Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 May/Jun;39(3):503-516.
doi: 10.1097/AUD.0000000000000502.

Validating a Method to Assess Lipreading, Audiovisual Gain, and Integration During Speech Reception With Cochlear-Implanted and Normal-Hearing Subjects Using a Talking Head

Stefan Schreitmüller  1 , Miriam Frenken  1 , Lüder Bentz  1 , Magdalene Ortmann  1 , Martin Walger  1   2 , Hartmut Meister  1
Affiliations

Validating a Method to Assess Lipreading, Audiovisual Gain, and Integration During Speech Reception With Cochlear-Implanted and Normal-Hearing Subjects Using a Talking Head

Stefan Schreitmüller et al. Ear Hear. 2018 May/Jun.

Abstract

Objectives: Watching a talker's mouth is beneficial for speech reception (SR) in many communication settings, especially in noise and when hearing is impaired. Measures for audiovisual (AV) SR can be valuable in the framework of diagnosing or treating hearing disorders. This study addresses the lack of standardized methods in many languages for assessing lipreading, AV gain, and integration. A new method is validated that supplements a German speech audiometric test with visualizations of the synthetic articulation of an avatar that was used, for it is feasible to lip-sync auditory speech in a highly standardized way. Three hypotheses were formed according to the literature on AV SR that used live or filmed talkers. It was tested whether respective effects could be reproduced with synthetic articulation: (1) cochlear implant (CI) users have a higher visual-only SR than normal-hearing (NH) individuals, and younger individuals obtain higher lipreading scores than older persons. (2) Both CI and NH gain from presenting AV over unimodal (auditory or visual) sentences in noise. (3) Both CI and NH listeners efficiently integrate complementary auditory and visual speech features.

Design: In a controlled, cross-sectional study with 14 experienced CI users (mean age 47.4) and 14 NH individuals (mean age 46.3, similar broad age distribution), lipreading, AV gain, and integration of a German matrix sentence test were assessed. Visual speech stimuli were synthesized by the articulation of the Talking Head system "MASSY" (Modular Audiovisual Speech Synthesizer), which displayed standardized articulation with respect to the visibility of German phones.

Results: In line with the hypotheses and previous literature, CI users had a higher mean visual-only SR than NH individuals (CI, 38%; NH, 12%; p < 0.001). Age was correlated with lipreading such that within each group, younger individuals obtained higher visual-only scores than older persons (rCI = -0.54; p = 0.046; rNH = -0.78; p < 0.001). Both CI and NH benefitted by AV over unimodal speech as indexed by calculations of the measures visual enhancement and auditory enhancement (each p < 0.001). Both groups efficiently integrated complementary auditory and visual speech features as indexed by calculations of the measure integration enhancement (each p < 0.005).

Conclusions: Given the good agreement between results from literature and the outcome of supplementing an existing validated auditory test with synthetic visual cues, the introduced method can be considered an interesting candidate for clinical and scientific applications to assess measures important for AV SR in a standardized manner. This could be beneficial for optimizing the diagnosis and treatment of individual listening and communication disorders, such as cochlear implantation.

PubMed Disclaimer

References

    1. Altieri N. A., Pisoni D. B., Townsend J. T.Some normative data on lip-reading skills (L). J Acoust Soc Am, 2011). 130, 14.
    1. Arnold P., Hill FBisensory augmentation: A speechreading advantage when speech is clearly audible and intact. Br J Psychol, 2001). 92(part 2), 339355.
    1. Auer E. T. Jr, Bernstein L. E.Enhanced visual speech perception in individuals with early-onset hearing impairment. J Speech Lang Hear Res, 2007). 50, 11571165.
    1. Bavelier D., Dye M. W., Hauser P. C.Do deaf individuals see better? Trends Cogn Sci, 2006). 10, 512518.
    1. Bergeson T. R., Pisoni D. B., Davis R. A.Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants. Ear Hear, 2005). 26, 149164.

Reference Notes

    1. Meister H., Schreitmüller S., Pyschny V., et al.Untersuchungen zur audiovisuellen Sprachwahrnehmung bei Personen mit Hörstörungen mittels virtuellem Kopfmodell. 2011). [Examining audiovisual speech reception with hearing impaired individuals using a virtual head model] Proceedings of the 14th Annual Convention of the German Association of Audiology (DGA) on CD-ROM. ISBN 978-3-9813141-1-3. A zip file is downloadable under https://urldefense.proofpoint.com/v2/url?u=http-3A__www.uzh.ch_orl_dga-2... where the respective report can be found in the folder “data” and its Filename is “Meister.pdf”.

Publication types