Contributions of fundamental frequency and timbre to vocal emotion perception and their electrophysiological correlates

Abstract

Our ability to infer a speaker's emotional state depends on the processing of acoustic parameters such as fundamental frequency (F0) and timbre. Yet, how these parameters are processed and integrated to inform emotion perception remains largely unknown. Here we pursued this issue using a novel parameter-specific voice morphing technique to create stimuli with emotion modulations in only F0 or only timbre. We used these stimuli together with fully modulated vocal stimuli in an event-related potential (ERP) study in which participants listened to and identified stimulus emotion. ERPs (P200 and N400) and behavioral data converged in showing that both F0 and timbre support emotion processing but do so differently for different emotions: Whereas F0 was most relevant for responses to happy, fearful and sad voices, timbre was most relevant for responses to voices expressing pleasure. Together, these findings offer original insights into the relative significance of different acoustic parameters for early neuronal representations of speaker emotion and show that such representations are predictive of subsequent evaluative judgments.

Keywords: event-related potentials (ERPs); fundamental frequency (F0); parameter-specific voice morphing; timbre; vocal emotion perception.

Fig. 1.

Schematic illustration of the different parameter-specific voice morphs.

Fig. 2.

Mean proportion of correct responses per Emotion and Morph Type.

Fig. 3.

Confusion data for each Emotion separately for the three Morph Types.

Fig. 4.

Scalp topographies of the contrast between the difference waves Diff_Full-F0 and Diff_Full-Timbre for each emotion separately from 50 to 500 ms.

Fig. 5.

ERPs separately for Emotion and Morph Type, averaged across nine channels.