Opposing serial dependencies revealed for sequences of auditory emotional stimuli

Abstract

Our percept of the world is not solely determined by what we perceive and process at a given moment in time, but also depends on what we processed recently. In the present study, we investigate whether the perceived emotion of a spoken sentence is contingent upon the emotion of an auditory stimulus on the preceding trial (i.e., serial dependence). Thereto, participants were exposed to spoken sentences that varied in emotional affect by changing the prosody that ranged from 'happy' to 'fearful'. Participants were instructed to rate the emotion. We found a positive serial dependence for emotion processing whereby the perceived emotion was biased towards the emotion on the preceding trial. When we introduced 'no-go' trials (i.e., no rating was required), we found a negative serial dependence when participants knew in advance to withhold their response on a given trial (Experiment 2) and a positive serial dependence when participants received the information to withhold their response after the stimulus presentation (Experiment 3). We therefore established a robust serial dependence for emotion processing in speech and introduce a methodology to disentangle perceptual from post-perceptual processes. This approach can be applied to the vast majority of studies investigating sequential dependencies to separate positive from negative serial dependence.

Keywords: audition; emotions in speech; go–no-go task; response bias; serial dependencies.

Figure 1.

Illustration of the procedure in Experiments 1-3. Participants were either instructed to respond to the auditory stimulus (a go trial) or to withhold their response (a no-go trial). In Experiments 1 and 2, participants knew in advance whether a response on a given trial was required before the auditory stimulus appeared. In Experiment 1, participants were instructed to respond to the auditory stimulus on every trial. In Experiment 2, participants were instructed that the trial types would alternate between go-trials (respond to the auditory stimulus) and no-go trials (withhold response). A go/no-go cue was provided to remind the participant of the next trial type. In Experiment 3, participants received a neutral cue prior to the presentation of the auditory stimulus. After the auditory stimulus, participants were informed about whether a response was required or not by presenting a go or no-go cue after 0 or 1000 ms delay. The trial type (go or no-go) was randomly determined on every trial. (Colour online).

Figure 2.

Results of Experiment 1. (a) Mean valence rating as a function of the auditory emotion on a given trial t. (b) Mean valence rating as a function of the auditory emotion on trial t − 1. (c) Mean valence rating as a function of the auditory emotion on trial t − 2. In panel a-c, the emotion on the x-axis (i.e., the actual stimulus) was either fearful (negative numbers) or happy (positive numbers). (d) Mean valence rating as a function of the auditory emotion on trial t − 1 (blue bars on the left) and trial t − 2 (green bars on the right). Here the light blue (collapsed over the light blue squares in Figure 2b) and the light green (collapsed over the light green squares in Figure 2c) bars represent a fearful emotion on trial t − 1 and trial t − 2, respectively. The dark blue (collapsed over the dark blue circles in Figure 2b) and the dark green (collapsed over the dark green circles in Figure 2c) bars signify a happy emotion on trial t − 1 and trial t − 2, respectively. In all panels, the error bars indicate ±1 standard error of the mean. (Colour online).

Figure 3.

Results of Experiment 2. (a) Mean valence rating on go trials as a function of the auditory emotion on a given trial t. (b) Mean valence rating on go trials as a function of the auditory emotion on trial t − 1 (no-go trial). (c) Mean valence rating on go trials as a function of the auditory emotion on trial t − 2 (go trial). In panel a-c, the emotion on the x-axis (i.e., the actual stimulus) was either fearful (negative numbers) or happy (positive numbers). (d) Mean valence rating on go trials as a function of the auditory emotion on trial t − 1 (blue bars on the left) and trial t − 2 (green bars on the right). Here the light blue (collapsed over the light blue squares in Figure 3b) and the light green (collapsed over the light green squares in Figure 3c) bars represent a fearful emotion on trial t − 1 and trial t − 2, respectively. The dark blue (collapsed over the dark blue circles in Figure 3b) and the dark green (collapsed over the dark green circles in Figure 3c) bars signifies a happy emotion on trial t − 1 and trial t − 2, respectively. Error bars indicate ±1 standard error of the mean. (Colour online).

Figure 4.

Results of Experiment 3. (a) Mean valence rating as a function of the auditory emotion on a given trial t. The emotion on the x-axis (i.e., the actual stimulus) was either fearful (negative numbers) or happy (positive numbers). (b and c) Mean valence rating (collapsed over all auditory test sentences) as a function of the emotion of the auditory stimulus on the previous trial (t − 1) and the delay to respond to the emotion on the preceding trial for go and no-go trials on t − 1, respectively. Error bars indicate the standard error of the mean. (Colour online).