Neural representations of emotion are organized around abstract event features

Abstract

Research on emotion attribution has tended to focus on the perception of overt expressions of at most five or six basic emotions. However, our ability to identify others' emotional states is not limited to perception of these canonical expressions. Instead, we make fine-grained inferences about what others feel based on the situations they encounter, relying on knowledge of the eliciting conditions for different emotions. In the present research, we provide convergent behavioral and neural evidence concerning the representations underlying these concepts. First, we find that patterns of activity in mentalizing regions contain information about subtle emotional distinctions conveyed through verbal descriptions of eliciting situations. Second, we identify a space of abstract situation features that well captures the emotion discriminations subjects make behaviorally and show that this feature space outperforms competing models in capturing the similarity space of neural patterns in these regions. Together, the data suggest that our knowledge of others' emotions is abstract and high dimensional, that brain regions selective for mental state reasoning support relatively subtle distinctions between emotion concepts, and that the neural representations in these regions are not reducible to more primitive affective dimensions such as valence and arousal.

Figure 1. Classification Results

(A) Above-chance 20-way classification of emotions in all ToM regions. (B)Whole-brain random-effects analysis of ToM localizer (FB > FP, red); searchlight map for 20-way emotion classification (blue); overlap (purple). (C) Classification accuracy broken down by emotion: average classification accuracy for each emotion condition (±SEM across exemplars) in behavioral judgments. (D) Correlation between behavioral classification accuracies (from C) and neural classification accuracies for each emotion class (based on errors of an SVM trained and tested on MMPFC voxel patterns).

Figure 2. Competing Behavioral Feature Spaces Derived from MTurk Ratings

(A–C) Matrix of emotions × average dimension scores for the appraisal space (A), the six basic emotion space (B), and the circumplex space (C). (D) Classification of 20 emotions (across stimulus exemplars) using information from each of the three competing spaces (±SEM across exemplars). Orange dotted line reflects chance (.05); blue dotted line reflects behavioral performance (.65).

Figure 3. RSA Methods

Representational dissimilarity matrices (RDMs) encode the pairwise Euclidean distances between different emotions within each feature space. For each region, a neural RDM captures the pairwise Euclidean distances between different emotions in the patterns of activity elicited across voxels (DMPFC shown here). Feature spaces are fit to the neural data by computing correlations between feature space RDMs and neural RDMs for each region in each subject. In addition to the three candidate theories, we also test confusion and categorical spaces. Given that the appraisal space best captures the distinctions between the 20 emotions, it could outperform simpler models simply by virtue of its superior emotion discrimination. To test this possibility, we compare the appraisal space to a pure categorical RDM, which assumes that all emotions are perfectly and equally discriminable. As a more conservative test, we compute the correlation between neural RDMs and the raw behavioral confusion matrix. Like the categorical model, this confusion RDM captures the distinctions between the 20 emotions but also encodes similarity between different emotions as reflected in the behavioral confusions. If the appraisal space outperforms these two models, it suggests that the appraisal space fits the neural data in virtue of the features rather than emotion discriminability alone.

Figure 4. RSA Results

Mean correlation (Kendall's tau) between candidate model RDMs and individual subject neural RDMs (±SEM across subjects). Dotted line shows the noise ceiling (see Table S3).