Aging and working memory modulate the ability to benefit from visible speech and iconic gestures during speech-in-noise comprehension

Abstract

When comprehending speech-in-noise (SiN), younger and older adults benefit from seeing the speaker's mouth, i.e. visible speech. Younger adults additionally benefit from manual iconic co-speech gestures. Here, we investigate to what extent younger and older adults benefit from perceiving both visual articulators while comprehending SiN, and whether this is modulated by working memory and inhibitory control. Twenty-eight younger and 28 older adults performed a word recognition task in three visual contexts: mouth blurred (speech-only), visible speech, or visible speech + iconic gesture. The speech signal was either clear or embedded in multitalker babble. Additionally, there were two visual-only conditions (visible speech, visible speech + gesture). Accuracy levels for both age groups were higher when both visual articulators were present compared to either one or none. However, older adults received a significantly smaller benefit than younger adults, although they performed equally well in speech-only and visual-only word recognition. Individual differences in verbal working memory and inhibitory control partly accounted for age-related performance differences. To conclude, perceiving iconic gestures in addition to visible speech improves younger and older adults' comprehension of SiN. Yet, the ability to benefit from this additional visual information is modulated by age and verbal working memory. Future research will have to show whether these findings extend beyond the single word level.

Fig. 1

Experimental overview. a Overview of conditions. Action words are in Dutch: lopen (“to walk”), fietsen (“to cycle”), rijden (“to drive”). b Trial structure. Answer options are in Dutch: strijden (“to fight”, phonological competitor), sturen (“to steer”, semantic competitor), afgieten (“to drain”, unrelated foil), rijden (“to drive”, target)

Fig. 2

Response accuracy in percent per age group and condition. Error bars represent SE. The dotted line separates the audiovisual trials (left) from the visual-only trials (right). ***p < 0.001, **p < 0.01, *p < 0.05

Fig. 3

Relative benefit per age group, noise level, and benefit type. The black line represents the median; the two hinges represent the 1st and 3rd quartile; the whiskers capture the largest and smallest observation but extend no further than 1.5 * IQR (data points outside 1.5 * IQR are represented by dots)