Computational approaches to the neuroscience of social perception

Abstract

Across multiple domains of social perception-including social categorization, emotion perception, impression formation and mentalizing-multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has permitted a more detailed understanding of how social information is processed and represented in the brain. As in other neuroimaging fields, the neuroscientific study of social perception initially relied on broad structure-function associations derived from univariate fMRI analysis to map neural regions involved in these processes. In this review, we trace the ways that social neuroscience studies using MVPA have built on these neuroanatomical associations to better characterize the computational relevance of different brain regions, and discuss how MVPA allows explicit tests of the correspondence between psychological models and the neural representation of social information. We also describe current and future advances in methodological approaches to multivariate fMRI data and their theoretical value for the neuroscience of social perception.

Keywords: computational neuroscience; emotion; multivariate pattern analysis; social cognition; social perception.

Fig. 1.

MVPA shows that stereotypes and emotion concepts shape representations of other people’s faces in the FG. Stolier and Freeman (2016) used multiple regression RSA on fMRI data from a task in which subjects viewed faces, showing that stereotypes partially structure how face’s social categories are represented in regions important for face perception such as the FG. (a) An example pair of corresponding DMs, depicting the corresponding representational structures of social categories in both stereotypes and subjective face perception. (b) Results from a whole-brain searchlight analysis, which performed multiple-regression RSA at each searchlight sphere, measuring the correspondence between the subjective perceptual and neural DMs while controlling for three models of visual similarity. This analysis revealed that the right fusiform gyrus (rFG) and OFC represent social categories in a manner consistent with the influence of stereotypes on processing of faces’ social categories. (c) Similar results are shown from Brooks et al. (2019), which reported an fMRI study in which subjects passively viewed faces varying in emotion expression. The researchers also measured subjects’ conceptual similarity between the emotion categories Anger, Disgust, Fear, Happiness, Sadness and Surprise (corresponding to the facial expressions shown in the scanner). The correspondence between this idiosyncratic conceptual DM and the brain’s representational structure (neural DM) was measured using multiple-regression RSA in a whole-brain searchlight analysis, also controlling for three visual similarity models. This analysis revealed that the rFG represents facial emotion categories in a manner consistent with the influence of a perceiver’s conceptual knowledge on processing of facial emotion. Figure adapted from Stolier and Freeman (2016) and Brooks et al. (2019).

Fig. 2.

MVPA sheds new light on how ‘social brain network’ regions represent other people. The ‘social brain network’, including regions such as the MPFC, precuneus/PCC, anterior temporal lobe (ATL) and TPJ, has long been known to be involved in social cognition and person perception. MVPA and RSA have been important tools in recent progress made in understanding how this network of regions specifically represents and computes social information. In a study by Thornton and colleagues (Thornton et al., 2019a), RSA was used to show that these regions represent individual identities in a manner consistent with the sum of a person’s mental state representations. Other studies have found that person and identity representations in these regions are structured by high-level social cognitive factors such as social network characteristics (Parkinson et al., 2017). In separate lines of work, MVPA and RSA have also proven helpful in disentangling the computational roles of these regions, e.g. suggesting that the MPFC is involved in representing information about individual people, while medial parietal regions such as the precuneus/PCC are more involved in representing information about the social context (Thornton and Mitchell, 2017). Figure adapted from Thornton and colleagues (Thornton et al., 2019b).