Endogenous variation in ventromedial prefrontal cortex state dynamics during naturalistic viewing reflects affective experience

Abstract

How we process ongoing experiences is shaped by our personal history, current needs, and future goals. Consequently, ventromedial prefrontal cortex (vmPFC) activity involved in processing these subjective appraisals appears to be highly idiosyncratic across individuals. To elucidate the role of the vmPFC in processing our ongoing experiences, we developed a computational framework and analysis pipeline to characterize the spatiotemporal dynamics of individual vmPFC responses as participants viewed a 45-minute television drama. Through a combination of functional magnetic resonance imaging, facial expression tracking, and self-reported emotional experiences across four studies, our data suggest that the vmPFC slowly transitions through a series of discretized states that broadly map onto affective experiences. Although these transitions typically occur at idiosyncratic times across people, participants exhibited a marked increase in state alignment during high affectively valenced events in the show. Our work suggests that the vmPFC ascribes affective meaning to our ongoing experiences.

Fig. 1. Temporal and spatial intersubject correlation.

(A) Temporal ISC for each region of interest (ROI) for study 1 (n = 13). Temporal ISC reflects the average pairwise correlation between each individual participant’s activation time series while viewing the 45-min episode. Heatmap blowouts depict all pairwise temporal ISCs between each participant for the vmPFC and V1 ROIs. L, left; R, right. (B) Temporal ISC for each ROI for study 2 (n = 35); same format as (A). (C) Spatial ISC across participants averaged across TRs for study 1. Spatial ISC reflects the pairwise spatial similarity of activity patterns across participants within each ROI at each TR. Functional alignment in the right subpanel was performed using the SRM (60) on an independent dataset. Parcels in which spatial ISC significantly increased following functional alignment are outlined in gray [estimated using a permutation test, false discovery rate (FDR) q < 0.05, which corresponds to P = 0.015]. (D) Spatial ISC at each TR across the 45-min episode for the vmPFC and V1 ROIs for study 1. Lighter colors denote anatomical alignment of voxels across participants. Darker colors denote functional alignment.

Fig. 2. Temporal recurrence of spatial patterns.

(A) A recurrence matrix of vmPFC spatial patterns for a representative participant. (B) A map of t tests over each pair of time points across all participants from study 2 (n = 35) thresholded using FDR q < 0.05 (vmPFC, P = 0.0007; V1, P = 0.011). (C) A subject-by-subject similarity matrix representing the consistency of spatiotemporal recurrence patterns across participants. Color bar is scaled between [−1, 1], note that the scaling on the t map depicted in (B) is one order of magnitude larger [−10, 10].

Fig. 3. Temporal autocorrelation.

(A) Left: The lag in TRs taken to reach an autocorrelation of 0.1 (arbitrarily selected). Larger values indicate more gradually changing voxel responses. Right: The median autocorrelation function across voxels for vmPFC (blue) and V1 (red) for each participant. The darker lines reflect an exponential function fit to the median of the distribution across voxels for each ROI, and the thin lines represent autocorrelation functions for each individual participant. (B) Identical analysis to (A) using spatial patterns across all voxels within each ROI, rather than for individual voxels.

Fig. 4. Individual-HMM estimation procedure.

(A) A schematic of how an HMM is fit to each participant’s ROI time series. We fit an HMM with a prespecified number of states to a principal components analysis (PCA)–reduced time series of voxel responses. This yielded estimates of the starting state, a transition matrix reflecting the likelihood of transitioning into any state at time t from time t − 1, and a pattern of activity (assumed to be an emission from an orthogonal multivariate Gaussian distribution). We plot the average aligned transition probabilities for study 2. We then applied the model to the same data to predict the most likely latent state at each moment in time using the Viterbi algorithm. (B) We aligned states across participants by maximizing the spatial similarity of the Gaussian emission weights associated with each estimated state using the Hungarian algorithm.

Fig. 5. Alignment of spatial patterns corresponding to latent states.

In this figure, we illustrate that the spatial patterns estimated from each individual HMM appear to be shared across participants. (A) We plot the spatial similarity of the emission patterns corresponding to each individual’s state after aligning the states across participants using the Hungarian algorithm. The outlines indicate patterns associated with each participant for a given state in vmPFC and V1 in study 1 (n = 13) and study 2 (n = 35). We highlight the vmPFC and V1 states with the highest intersubject spatial similarity (blue, study 1; red, study 2). (B) We plot the magnitude of the spatial similarity for the states with the highest overall spatial similarity highlighted in (A) averaged across participants. (C) We plot the vmPFC and V1 spatial patterns that correspond to the states associated with the highest intersubject spatial similarity from studies 1 and 2. These maps reflect the emission probabilities associated with each state averaged across participants. (D) Temporal concordance of the highlighted predicted states across individuals within each study. Higher values indicate a higher proportion of participants occupying the same putative latent state within each study (blue, study 1; red, study 2).

Fig. 6. Affective experiences associated with increased state concordance.

(A) Screenshots indicate specific scenes from the television show, FNL. These images are copyright of NBCUniversal, LLC. Five participants recorded watching the show using custom head-mounted cameras. These participants have provided consent to sharing their face expression data in this image. Photo credit: Jin Hyun Cheong, Dartmouth College. Line plots illustrate state concordance at each moment in time for each of the four states identified by the group vmPFC HMM (studies 1 and 2). We also plot the interexperiment latent components estimated using the SRM across these two studies and two additional behavioral studies (study 3, facial expression) and subjective ratings (study 4, self-reported feelings). (B) Here, we visualize the transformation matrices that project study 3 face expressions and study 4 subjective feelings into the interexperiment latent space. Darker colors in the face expressions indicate higher AU intensity. Darker and larger words in the word cloud indicate higher contributions of the feeling onto the latent component. (C) Overall interexperiment temporal similarity for each estimated latent component across all four studies. (D) Loadings of the vmPFC state concordances (studies 1 and 2) onto each interexperiment latent component. (E) Whole-brain univariate contrasts based on vmPFC state occupancy. We computed the average voxel-wise activation associated with each state compared to all other states based and plot the group-level results thresholded with FDR q < 0.05.