Common Functional Brain States Encode both Perceived Emotion and the Psychophysiological Response to Affective Stimuli

Abstract

Multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has critically advanced the neuroanatomical understanding of affect processing in the human brain. Central to these advancements is the brain state, a temporally-succinct fMRI-derived pattern of neural activation, which serves as a processing unit. Establishing the brain state's central role in affect processing, however, requires that it predicts multiple independent measures of affect. We employed MVPA-based regression to predict the valence and arousal properties of visual stimuli sampled from the International Affective Picture System (IAPS) along with the corollary skin conductance response (SCR) for demographically diverse healthy human participants (n = 19). We found that brain states significantly predicted the normative valence and arousal scores of the stimuli as well as the attendant individual SCRs. In contrast, SCRs significantly predicted arousal only. The prediction effect size of the brain state was more than three times greater than that of SCR. Moreover, neuroanatomical analysis of the regression parameters found remarkable agreement with regions long-established by fMRI univariate analyses in the emotion processing literature. Finally, geometric analysis of these parameters also found that the neuroanatomical encodings of valence and arousal are orthogonal as originally posited by the circumplex model of dimensional emotion.

Figure 1

Methodological Overview and Conceptual Model. (a) Experiment Design: Ninety images were sampled from the International Affective Picture System (IAPS) to form a subset that maximally spanned the affective properties of valence (v) and arousal (a). (b) Signal Acquisition: Image stimuli were presented for 2 s interleaved with random inter-trial intervals [2–6 s]; fMRI measurements of the blood oxygen level dependent (BOLD) response were recorded concurrently with the skin conductance response (SCR). (‡) Conceptual Model: We hypothesize that brain states, s, simultaneously encode the dimensional affective properties of their image stimuli as well as the attendant psychophysiological responses. (c) Brain and Physiological State Estimation: fMRI signals were preprocessed to remove noise and motion artifacts and segmented to remove all voxels except gray matter (GM); SCR signals were preprocessed to remove noise and tonic signal components; neural activation patterns were extracted for each stimulus according to the beta-series method; and, dimensionally reduced. (d) Prediction of Affective Signals: Intra-subject cross-validated linear support vector machine (SVM) regression was conducted on the beta-series (labeled according to the stimulus from which they were extracted). The figure depicts the regression model labeling the affective property of a novel point. (e) Effect Size Estimation: Group-level predictions of affective properties and measurements were conducted via General Linear Mixed-Effects Models (GLMMs) in three tests: (1) the measurements of interest were the normative affective properties of the stimuli (v, a) and the fixed effects were the SCR measurements of affect state (β^SCR); (2) the measurements of interest were the affective properties (v, a) and the fixed effects were the SVM-predicted properties (~v, ~a); and, (3) the measurements of interest were β^SCR and the fixed effects were the SVM-predicted affective responses (~β^SCR). (*) The individual SVM models of GM-based features were transformed into encoding representations of affect state and anatomically analyzed group-wise (not pictured). (**) GLMM random effects for slope and intercept were modeled subject-wise. Note, details of the experiment design, preprocessing pipeline, and brain state estimation methodology have been reported previously.

Figure 2

Summary of the primary experimental findings in relation to the proposed conceptual model. Affective properties of the IAPS imageset, valence (v) and arousal (a), brain state (s), and skin conductance responses (SCR) are depicted alongside arrows indicating the direction of significant GLMM-based predictions (^†p < 0.05, ^‡p < 0.001, F-test), reported in units of effect size (Pearson’s r).

Figure 3

Group-level gray matter mean intra-subject emotion perception encodings of normative affect properties. Color gradations indicate the group-level mean encoding parameter (red indicating positive valence or high arousal, blue indicating negative valence or low arousal). Only group-level significant parameters (p < 0.05, global permutation test) are depicted (see Materials and Methods: Neuroanatomical Encoding Parameter Significance via Permutation Testing). Image slices are presented in Talairach coordinate space and neurological convention. Maximum mean voxel intensity |μ(z)| = 2.0. Only those clusters having 10 or greater contiguous voxels (NN = 1) are plotted.

Figure 4

Group-level gray matter mean intra-subject encodings of individual SCRs. Color gradations indicate the group-level mean activation (red indicating high SCR, blue indicating low SCR). Only group-level significant parameters (p < 0.05, global permutation test) are depicted. Image slices are presented in Talairach coordinate space using the neurological convention. Maximum mean voxel intensity |μ(z)| = 2.0. Only those clusters having 10 or greater contiguous voxels (NN = 1) are plotted.

Figure 5

Quantitative group-level comparison of the cosine similarities within and between the neural activation encodings of arousal (a) and valence (v). Similarities are presented as boxplots; gray circle markers depict group-level mean similarity scores for each participant; horizontal red segments indicate the mean distribution values; upper and lower red filled areas represent the 95% confidence intervals of the means; and, blue filled areas represent the distributions’ first standard deviations. Neural encoding similarity is measured according to the cosine of the angle between the vectors describing the compared encoding models of the SVM hyperplanes (see Materials and Methods: Calculation of Cosine Similarity). For each participant, the group-level similarity is formed from the group average between the participant’s hyperplane and the comparison hyperplanes (excluding self-similarity).