The Timing and Effort of Lexical Access in Natural and Degraded Speech

Abstract

Understanding speech is effortless in ideal situations, and although adverse conditions, such as caused by hearing impairment, often render it an effortful task, they do not necessarily suspend speech comprehension. A prime example of this is speech perception by cochlear implant users, whose hearing prostheses transmit speech as a significantly degraded signal. It is yet unknown how mechanisms of speech processing deal with such degraded signals, and whether they are affected by effortful processing of speech. This paper compares the automatic process of lexical competition between natural and degraded speech, and combines gaze fixations, which capture the course of lexical disambiguation, with pupillometry, which quantifies the mental effort involved in processing speech. Listeners' ocular responses were recorded during disambiguation of lexical embeddings with matching and mismatching durational cues. Durational cues were selected due to their substantial role in listeners' quick limitation of the number of lexical candidates for lexical access in natural speech. Results showed that lexical competition increased mental effort in processing natural stimuli in particular in presence of mismatching cues. Signal degradation reduced listeners' ability to quickly integrate durational cues in lexical selection, and delayed and prolonged lexical competition. The effort of processing degraded speech was increased overall, and because it had its sources at the pre-lexical level this effect can be attributed to listening to degraded speech rather than to lexical disambiguation. In sum, the course of lexical competition was largely comparable for natural and degraded speech, but showed crucial shifts in timing, and different sources of increased mental effort. We argue that well-timed progress of information from sensory to pre-lexical and lexical stages of processing, which is the result of perceptual adaptation during speech development, is the reason why in ideal situations speech is perceived as an undemanding task. Degradation of the signal or the receiver channel can quickly bring this well-adjusted timing out of balance and lead to increase in mental effort. Incomplete and effortful processing at the early pre-lexical stages has its consequences on lexical processing as it adds uncertainty to the forming and revising of lexical hypotheses.

Keywords: cochlear implants; lexical processing; pupillometry; speech perception in adverse communicative situations; time-course of speech perception.

FIGURE 1

Examples of the stimuli for the two experimental conditions for natural speech (NS) and degraded speech (DS). Depicted are the waveforms and spectrograms of the natural stimuli (top), and the degraded stimuli (bottom), with target-matching (left) and target-mismatching duration cues (right).

FIGURE 2

Example of the display presented to the participants, with bok (goat) as competitor for the target bokser (boxer).

FIGURE 3

Curves of proportions of gaze fixation over time for the target-matching and target-mismatching conditions, when presented with natural (NS) and degraded speech (DS). The green lines show the proportion of fixations averaged across participants and items and the 95% confidence intervals for target fixations, red lines show the same for competitor fixations, and the dashed black lines show fixations to the distractors.

FIGURE 4

Effect of mismatching cues in NS (left) and DS (right). The probability of fixations toward the competitor in target matching (black), and target mismatching stimuli (red) is displayed for NS and DS conditions, in the averaged data (solid lines), and in the data fitted by the statistical model (dashed).

FIGURE 5

Pupil dilation data time curves shown for NS (left) and DS (right) for target matching (green), target-mismatching (red) and filler stimuli (black).

FIGURE 6

Effect of lexical competition on mental effort in NS (top) and DS (bottom), as captured by the difference in ERDP for critical experimental items and fillers. The functions displayed are smoothed to better represent the trend that is visible in the raw data displayed in Figure 5.

FIGURE 7

Effect of listening to the sentences on mental effort, pupil dilation curves over time relative to baseline 2.