Motion cues modulate responses to emotion in movies

Abstract

Film theorists and practitioners suggest that motion can be manipulated in movie scenes to elicit emotional responses in viewers. However, our understanding of the role of motion in emotion perception remains limited. On the one hand, movies continuously depict local motion- movements of objects and humans, which are crucial for generating emotional responses. Movie scenes also frequently portray global motion, mainly induced by large camera movements, global motion being yet another source of information used by the brain during natural vision. Here we used functional MRI to elucidate the contributions of local and global motion to emotion perception during movie viewing. Subjects observed long (1 min) movie segments depicting emotional or neutral content. Brain activity in areas that showed preferential responses to emotional content was strongly linked over time with frame-wide variations in global motion, and to a lesser extent with local motion information. Similarly, stronger responses to emotional content were recorded within regions of interest whose activity was attuned to global and local motion over time. Since global motion fields are experienced during self-motion, we suggest that camera movements may induce illusory self-motion cues in viewers that interact with the movie's narrative and with other emotional cues in generating affective responses.

Figure 1

Experimental design and stimuli. (A) Blood oxygenation level dependent (BOLD) responses were recorded while subjects observed 1-min long clips, interleaved by short periods of fixation, depicting either emotional or emotionally neutral content. The clips were extracted from popular movies (see Materials and Methods). Motion information contained in the clips was quantified by computing frame-wide changes in optical flow and partitioning these into changes in global and local motion fields. The global and local motion signals were tightly correlated across time (B) and were stronger during emotional than during emotionally neutral movie frames (C).

Figure 2

Brain activity associated with observation of emotion in movies. (A) Regions showing significantly stronger BOLD responses to emotional relative to neutral clips, FDR corrected for multiple comparisons. To assess the contribution of motion to the perception of emotion in movies the global (B) and local (C) motion time series were regressed with individual-subject BOLD signals extracted from each of the regions displayed in A, shown here for one representative region (color shades denote standard errors of the mean). (D) Brain activation in emotion responsive brain regions was strongly tuned to global and to a lesser degree to local motion. Asterisk signs within bars denote the results of one-sample t tests, indicating whether the regression coefficients were significantly different than ‘0’. Asterisk signs above bars denote the results of paired t tests, comparing links (regressions coefficients) between BOLD signals and global and local flow. Cb, cerebellum; FG, fusiform gyrus; IFG, inferior frontal gyrus; IOG, inferior occipital gyrus; Lg, lingual gyrus; MTG, medial temporal gyrus; Pcu, precuneus; STG, superior temporal gyrus; * indicates statistical significance at P < 0.05; **p < 0.01; ***P < 0.001. Significance estimates were FDR corrected for multiple comparisons.

Figure 3

Analysis of brain activity induced by global and local flow. (A,B) The global (A) and local (B) flow signals served as a predictors in a GLM analysis. The signals are displayed along with the corresponding segments where emotional and neutral clips were shown. (C,D) Regions whose activity was correlated with the global (C) and with the local (D) flow predictors are shown in orange alongside the contour of regions that showed significantly stronger BOLD responses to emotional relative to neutral clips (as displayed in Fig. 2). The map is FDR corrected (<0.05) for multiple comparisons (E) Estimates for the overlap between emotion-related brain activations and those induced by global and local flow (F,G) Region of Interest analysis (ROI) comparing brain activity elicited by emotional and neutral content within all the regions responsive to global (F) and local flow (G). *Indicates statistical significance at P < 0.05; **P < 0.01; ***P < 0.001. Cu, Cuneus; MFG, middle frontal gyrus; MOG, Middle occipital gyrus; PCg, Posterior cingulate gyrus; SFG, Superior frontal gyrus; Thal, Thalamus. All other abbreviations are as in previous figures.

Figure 4

Brain activation related to processing of discrete emotions in Region of Interest tuned to global and local flow. ROI analysis comparing brain activity elicited by anger and joy, relative to neutral content, was performed for all regions responsive to global (A,C) and local (B,D) flow. * indicates statistical significance at P < 0.05; **P < 0.01; ***P < 0.001. All abbreviations are as in previous figures.