Impact of Background Noise and Sentence Complexity on Processing Demands during Sentence Comprehension

Abstract

Speech comprehension in adverse listening conditions can be effortful even when speech is fully intelligible. Acoustical distortions typically make speech comprehension more effortful, but effort also depends on linguistic aspects of the speech signal, such as its syntactic complexity. In the present study, pupil dilations, and subjective effort ratings were recorded in 20 normal-hearing participants while performing a sentence comprehension task. The sentences were either syntactically simple (subject-first sentence structure) or complex (object-first sentence structure) and were presented in two levels of background noise both corresponding to high intelligibility. A digit span and a reading span test were used to assess individual differences in the participants' working memory capacity (WMC). The results showed that the subjectively rated effort was mostly affected by the noise level and less by syntactic complexity. Conversely, pupil dilations increased with syntactic complexity but only showed a small effect of the noise level. Participants with higher WMC showed increased pupil responses in the higher-level noise condition but rated sentence comprehension as being less effortful compared to participants with lower WMC. Overall, the results demonstrate that pupil dilations and subjectively rated effort represent different aspects of effort. Furthermore, the results indicate that effort can vary in situations with high speech intelligibility.

Keywords: background noise; digit span; effort; processing demands; pupillometry; reading span; syntactic complexity; working memory capacity.

FIGURE 1

Example of a visual stimulus pair used in the audio-visual picture-matching paradigm. The left figure shows a target picture corresponding to the sentences Den sure pingvin vil filme den søde koala (“The angry penguin will film the sweet koala”; SVO I in Table 1) or Den søde koala vil den sure pingvin filme (“The sweet koala, the angry penguin will film.”; OVS II in Table 1). The right figure shows an example for the corresponding competitor picture of the same sentences. Only one of the pictures, either target or competitor picture, was presented during the paradigm.

FIGURE 2

Trial structure of the audio-visual picture-matching paradigm. Participants saw a picture on screen for 2000 ms, followed by a visual fixation cross and a simultaneous acoustical presentation of a sentence in background noise. Background noise was presented 3000 ms before and ended 3000 ms after sentence offset. After the acoustic presentation, participants’ task was to decide whether the picture matched with the sentence or not. Pupil dilations were measured from the picture onset until the participants’ response in the comprehension task. The comprehension task was followed by a subjective rating of the experienced difficulty.

FIGURE 3

Normalized pupil dilation averaged across all participants for all four conditions. Time axis starts with the onset of sentence presentation. Horizontal lines indicated interval used for baseline correction and the different epochs in which the mean pupil response was calculated.

FIGURE 4

(Left) Subjectively rated difficulty averaged across participants for complex (OVS) and simple (SVO) sentence structures presented at higher-level noise (black) and lower-level noise (white) conditions. (Right) Response accuracies averaged across all participants and trials for complex (OVS) and simple (SVO) sentences presented at the higher-level noise (black) and the lower-level noise (white) conditions. The error bars show the standard errors.

FIGURE 5

Mean pupil dilation observed for all four conditions. Time-averaged pupil dilation was calculated for three different epochs. Epoch 1 is the time when the first part of the sentence was presented. Epoch 2 includes the time after the sentence was disambiguated until the comprehension question. The third epoch is defined as the time from sentence onset until participants’ response. The error bars show the standard deviations.

FIGURE 6

(Left) Digit span scores as a function of subjective ratings. (Middle) Digit span scores as a function of the of the pupil dilation in the audio-visual picture-matching paradigm. (Right) Pupil dilations in the digit span test as a function of the pupil dilation in the audio-visual picture-matching paradigm.