Audiovisual Representations of Valence: a Cross-study Perspective

Abstract

Hedonic valence describes the pleasantness or unpleasantness of psychological states elicited by stimuli and is conceived as a fundamental building block of emotional experience. Multivariate pattern analysis approaches contribute to the study of valence representation by allowing identification of valence from distributed patterns of activity. However, the issue of construct validity arises in that there is always a possibility that classification results from a single study are driven by factors other than valence, such as the idiosyncrasies of the stimuli. In this work, we identify valence across participants from six different fMRI studies that used auditory, visual, or audiovisual stimuli, thus increasing the likelihood that classification is driven by valence and not by the specifics of the experimental paradigm of a particular study. The studies included a total of 93 participants and differed on stimuli, task, trial duration, number of participants, and scanner parameters. In a leave-one-study-out cross-validation procedure, we trained the classifiers on fMRI data from five studies and predicted valence, positive or negative, for each of the participants in the left-out study. Whole-brain classification demonstrated a reliable distinction between positive and negative valence states (72% accuracy). In a searchlight analysis, the representation of valence was localized to the right postcentral and supramarginal gyri, left superior frontal and middle frontal cortices, and right pregenual anterior cingulate and superior medial frontal cortices. The demonstrated cross-study classification of valence enhances the construct validity and generalizability of the findings from the combined studies.

Keywords: MVPA; Valence; fMRI.

Fig. 1

Whole-brain classification accuracies with leave-one-study-out cross-validation are shown with standard errors across cross-validation folds for different numbers of voxels. Dash line indicates the chance level (0.5)

Fig. 2

Clusters sensitive to valence identified by searchlight analysis with leave-one-study-out cross-validation. Supramarginal (SM); postcentral (PC); superior frontal cortex (SFC); middle frontal cortex (MFC); superior medial prefrontal cortex (MPFC); pregenual anterior cingulate cortex (pgACC)

Fig. 3

Clusters sensitive to cross-modal valence identified by searchlight analysis with leave-one-participant-out cross-validation