Subjective and psychophysiological indexes of listening effort in a competing-talker task

Abstract

Background: The effects of noise and other competing backgrounds on speech recognition performance are well documented. There is less information, however, on listening effort and stress experienced by listeners during a speech-recognition task that requires inhibition of competing sounds.

Purpose: The purpose was (a) to determine if psychophysiological indexes of listening effort were more sensitive than performance measures (percentage correct) obtained near ceiling level during a competing speech task, (b) to determine the relative sensitivity of four psychophysiological measures to changes in task demand, and (c) to determine the relationships between changes in psychophysiological measures and changes in subjective ratings of stress and workload.

Research design: A repeated-measures experimental design was used to examine changes in performance, psychophysiological measures, and subjective ratings in response to increasing task demand.

Study sample: Fifteen adults with normal hearing participated in the study. The mean age of the participants was 27 (range: 24-54).

Data collection and analysis: Psychophysiological recordings of heart rate, skin conductance, skin temperature, and electromyographic (EMG) activity were obtained during listening tasks of varying demand. Materials from the Dichotic Digits Test were used to modulate task demand. The three levels of task demand were single digits presented to one ear (low-demand reference condition), single digits presented simultaneously to both ears (medium demand), and a series of two digits presented simultaneously to both ears (high demand). Participants were asked to repeat all the digits they heard, while psychophysiological activity was recorded simultaneously. Subjective ratings of task load were obtained after each condition using the National Aeronautics and Space Administration Task Load Index questionnaire. Repeated-measures analyses of variance were completed for each measure using task demand and session as factors.

Results: Mean performance was higher than 96% for all listening tasks. There was no significant change in performance across listening conditions for any listener. There was, however, a significant increase in mean skin conductance and EMG activity as task demand increased. Heart rate and skin temperature did not change significantly. There was no strong association between subjective and psychophysiological measures, but all participants with mean normalized effort ratings of greater than 4.5 (i.e., effort increased by a factor of at least 4.5) showed significant changes in skin conductance.

Conclusions: Even in the absence of substantial performance changes, listeners may experience changes in subjective and psychophysiological responses consistent with the activation of a stress response. Skin conductance appears to be the most promising measure for evaluating individual changes in psychophysiological responses during listening tasks.

Figure 1

Mean NASA-TLX ratings of mental demand, effort, perceived error, and frustration/stress for the low-, medium-, and high-demand tasks. Recognition scores are also shown. The error bars denote ± 1 standard error.

Figure 2

Mean EMG, skin conductance, skin temperature, and heart rate measures for the low-, medium-, and high-demand tasks. The error bars denote ± 1 standard error.

Figure 3

Relationship between mean normalized effort ratings and skin conductance for all participants (top panel) and for 14 of the 15 participants (outlier excluded). The circles denote participants who had mean skin conductance z-scores of greater than 2.0 or less than −2.0 (significant change). The hatched squares indicate participants whose changes in skin conductance were not significant.