Aging, audiovisual integration, and the principle of inverse effectiveness

Abstract

Objective: The purpose of this investigation was to compare the ability of young and older adults to integrate auditory and visual sentence materials under conditions of good and poor signal clarity. The principle of inverse effectiveness (PoIE), which characterizes many neuronal and behavioral phenomena related to multisensory integration, asserts that as unimodal performance declines, integration is enhanced. Thus, the PoIE predicts that both young and older adults will show enhanced integration of auditory and visual speech stimuli when these stimuli are degraded. More importantly, because older adults' unimodal speech recognition skills decline in both the auditory and visual domains, the PoIE predicts that older adults will show enhanced integration during audiovisual speech recognition relative to younger adults. This study provides a test of these predictions.

Design: Fifty-three young and 53 older adults with normal hearing completed the closed-set Build-A-Sentence test and the CUNY Sentence test in a total of eight conditions; four unimodal and four audiovisual. In the unimodal conditions, stimuli were either auditory or visual and either easier or harder to perceive; the audiovisual conditions were formed from all the combinations of the unimodal signals. The hard visual signals were created by degrading video contrast, and the hard auditory signals were created by decreasing the signal to noise ratio. Scores from the unimodal and bimodal conditions were used to compute auditory enhancement and integration enhancement measures.

Results: Contrary to the PoIE, neither the auditory enhancement nor integration enhancement measures increased when signal clarity in the auditory or visual channel of audiovisual speech stimuli was decreased, nor was either measure higher for older adults than for young adults. In audiovisual conditions with easy visual stimuli, the integration enhancement measure for older adults was equivalent to that for young adults. However, in conditions with hard visual stimuli, integration enhancement for older adults was significantly lower than that for young adults.

Conclusions: The present findings do not support extension of the PoIE to audiovisual speech recognition. Our results are not consistent with either the prediction that integration would be enhanced under conditions of poor signal clarity or the prediction that older adults would show enhanced integration, relative to young adults. Although there is a considerable controversy with regard to the best way to measure audiovisual integration, the fact that two of the most prominent measures, auditory enhancement and integration enhancement, both yielded results inconsistent with the PoIE, strongly suggests that the integration of audiovisual speech stimuli differs in some fundamental way from the integration of other bimodal stimuli. The results also suggest that aging does not impair integration enhancement when the visual speech signal has good clarity, but may affect it when the visual speech signal has poor clarity.

Figure 1

Mean percentage correct for the young and older participants in the four unimodal conditions for the BAS Test. Error bars indicate the standard error of the mean.

Figure 2

Mean percentage correct for the young and older participants in the four audiovisual conditions for the BAS Test. Error bars indicate the standard error of the mean.

Figure 3

Mean auditory enhancement (AE) for the young and older participants in the four audiovisual conditions for the BAS Test. Error bars indicate the standard error of the mean.

Figure 4

Mean integration enhancement (IE) for the young and older participants in the four audiovisual conditions for the BAS Test. Error bars indicate the standard error of the mean.

Figure 5

Mean percent correct scores for the young and older participants in the four unimodal conditions for the CUNY Sentence Test. Error bars indicate the standard error of the mean.

Figure 6

Mean percent correct scores for the young and older participants in the four audiovisual conditions for the CUNY Sentence Test. Error bars indicate the standard error of the mean.

Figure 7

Mean auditory enhancement (AE) for the young and older participants in the four audiovisual conditions for the CUNY Sentence Test. Error bars indicate the standard error of the mean.

Figure 8

Mean integration enhancement (IE) for the young and older participants in the four audiovisual conditions for the CUNY Sentence Test. Error bars indicate the standard error of the mean.