Pupil Dilation Is Sensitive to Semantic Ambiguity and Acoustic Degradation

Abstract

Speech comprehension is challenged by background noise, acoustic interference, and linguistic factors, such as the presence of words with more than one meaning (homonyms and homophones). Previous work suggests that homophony in spoken language increases cognitive demand. Here, we measured pupil dilation-a physiological index of cognitive demand-while listeners heard high-ambiguity sentences, containing words with more than one meaning, or well-matched low-ambiguity sentences without ambiguous words. This semantic-ambiguity manipulation was crossed with an acoustic manipulation in two experiments. In Experiment 1, sentences were masked with 30-talker babble at 0 and +6 dB signal-to-noise ratio (SNR), and in Experiment 2, sentences were heard with or without a pink noise masker at -2 dB SNR. Speech comprehension was measured by asking listeners to judge the semantic relatedness of a visual probe word to the previous sentence. In both experiments, comprehension was lower for high- than for low-ambiguity sentences when SNRs were low. Pupils dilated more when sentences included ambiguous words, even when no noise was added (Experiment 2). Pupil also dilated more when SNRs were low. The effect of masking was larger than the effect of ambiguity for performance and pupil responses. This work demonstrates that the presence of homophones, a condition that is ubiquitous in natural language, increases cognitive demand and reduces intelligibility of speech heard with a noisy background.

Keywords: cognitive load; listening effort; pupillometry; semantic ambiguity; speech masking.

Figure 1.

Experimental Designs for Experiments 1 and 2. Schematic timeline of a trial in Experiment 1 (A) and Experiment 2 (B). A trial started 3 s prior to sentence onset with a visual fixation ring (and in Experiment 1 with the onset of the background babble noise). A probe word was presented visually 1.2 s after sentence offset. Participants were asked to indicate whether the probe word was semantically related or unrelated to the sentence.

Figure 2.

Experiment 1: Proportion Correct in Semantic-Relatedness Task. Mean proportion of correct responses for each condition. Error bars reflect the standard error of the mean. The Clarity × Ambiguity interaction was significant. LA6 = low ambiguity in +6 dB SNR babble; HA6 = high ambiguity in +6 dB SNR babble; LA0 = low ambiguity in 0 dB SNR babble; HA0 = high ambiguity in 0 dB SNR babble.

Figure 3.

Experiment 1: Pupil Dilation Results. (A) Time course of pupil area (averaged across participants; N = 38). (B) Mean pupil area from 0.5 s after sentence onset to 1 s after sentence offset. (C) Peak pupil area. (D) Peak pupil latency. Error bars reflect the standard error of the mean. (E) Individual data scatterplot for Ambiguity main effect (mean pupil area). (F) Individual scatterplot for Clarity main effect (mean pupil area). LA6 = low ambiguity in +6 dB SNR babble; HA6 = high ambiguity in +6 dB SNR babble; LA0 = low ambiguity in 0 dB SNR babble; HA0 = high ambiguity in 0 dB SNR babble; SNR = signal-to-noise ratio.

Figure 4.

Experiment 2: Proportion Correct in Semantic-Relatedness Task. Mean proportion of correct responses for each condition. Error bars reflect the standard error of the mean. The Clarity × Ambiguity interaction was significant. LAC = low ambiguity in clear; HAC = high ambiguity in clear; LAN = low ambiguity in –2 dB SNR pink noise; HAN = high ambiguity in –2 dB SNR pink noise.

Figure 5.

Experiment 2: Pupil Dilation Results. (A) Time course of pupil area (averaged across participants; N = 35). (B) Mean pupil area from 0.5 s after sentence onset to 1 s after sentence offset. (C) Peak pupil area. (D) Peak pupil latency. Error bars in (B), (C), and (D) reflect the standard error of the mean. (E) Individual data scatterplot for the Ambiguity main effect (mean pupil area). (F) Individual data scatterplot for Clarity main effect (mean pupil area). LAC = low ambiguity in clear; HAC = high ambiguity in clear; LAN = low ambiguity in –2 dB SNR pink noise; HAN = high ambiguity in –2 dB SNR pink noise.

Figure 6.

Results for Microsaccade Analysis. Time courses of microsaccade rate for Experiment 1 (A) and Experiment 2 (B). Bar graphs show the mean microsaccade rate for each condition for Experiment 1 (C) and Experiment 2 (D). Error bars reflect the standard error of the mean. MS = microsaccade; LA6 = low ambiguity in +6 dB SNR babble; HA6 = high ambiguity in +6 dB SNR babble; LA0 = low ambiguity in 0 dB SNR babble; HA0 = high ambiguity in 0 dB SNR babble; LAC = low ambiguity in clear; HAC = high ambiguity in clear; LAN = low ambiguity in –2 dB SNR pink noise; HAN = high ambiguity in –2 dB SNR pink noise.