Measures of Listening Effort Are Multidimensional

Abstract

Objectives: Listening effort can be defined as the cognitive resources required to perform a listening task. The literature on listening effort is as confusing as it is voluminous: measures of listening effort rarely correlate with each other and sometimes result in contradictory findings. Here, we directly compared simultaneously recorded multimodal measures of listening effort. After establishing the reliability of the measures, we investigated validity by quantifying correlations between measures and then grouping-related measures through factor analysis.

Design: One hundred and sixteen participants with audiometric thresholds ranging from normal to severe hearing loss took part in the study (age range: 55 to 85 years old, 50.3% male). We simultaneously measured pupil size, electroencephalographic alpha power, skin conductance, and self-report listening effort. One self-report measure of fatigue was also included. The signal to noise ratio (SNR) was adjusted at 71% criterion performance using sequences of 3 digits. The main listening task involved correct recall of a random digit from a sequence of six presented at a SNR where performance was around 82 to 93%. Test-retest reliability of the measures was established by retesting 30 participants 7 days after the initial session.

Results: With the exception of skin conductance and the self-report measure of fatigue, interclass correlation coefficients (ICC) revealed good test-retest reliability (minimum ICC: 0.71). Weak or nonsignificant correlations were identified between measures. Factor analysis, using only the reliable measures, revealed four underlying dimensions: factor 1 included SNR, hearing level, baseline alpha power, and performance accuracy; factor 2 included pupillometry; factor 3 included alpha power (during speech presentation and during retention); factor 4 included self-reported listening effort and baseline alpha power.

Conclusions: The good ICC suggests that poor test reliability is not the reason for the lack of correlation between measures. We have demonstrated that measures traditionally used as indicators of listening effort tap into multiple underlying dimensions. We therefore propose that there is no "gold standard" measure of listening effort and that different measures of listening effort should not be used interchangeably. When choosing method(s) to measure listening effort, the nature of the task and aspects of increased listening demands that are of interest should be taken into account. The findings of this study provide a framework for understanding and interpreting listening effort measures.

Fig. 1.

An outline of the sequence of events in each trial and the time periods used when analyzing the data for each measure. Retention: the period during which participants memorized the digits; recovery: the period before the start of a new trial. B indicates baseline period; EEG, electroencephalography.

Fig. 2.

Mean change in alpha power across participants and trials. The temperature scale represents changes in event-related band power in decibels (dB). The top panel shows changes in alpha activity during the retention period relative to baseline alpha activity in the recovery period, that is, before the noise is presented. The bottom panel shows changes in alpha activity during the speech presentation period relative to alpha activity during the last second of unmodulated noise, that is, the period of noise alone that preceded the presentation of the first spoken digit. Dashed boxes represent the time periods of increased alpha activity (n =116). The X-axis represents time.

Fig. 3.

Associations between the test (x axis) and retest (y axis) data (n = 30). NASA indicates National Aeronautics and Space Administration; VAS_F, Visual Analogue Scale of Fatigue.

Fig. 4.

Mean change in pupil size relative to baseline across participants and trials. The black line represents mean change in pupil size across participants and trials (y) axis. The shaded gray area represents ±1 SE. (n = 116). Time in seconds (0 to 4 sec: speech presentation period, 4 to 7 sec: retention period) is shown on the x axis.

Fig. 5.

Scree plot and extracted eigenvalues.