The Kuleshov Effect: the influence of contextual framing on emotional attributions

Abstract

Filmmakers have long recognized the importance of editing techniques to guide the audiences' perceptions and enhance the impact of a scene. We demonstrate behaviorally that pairing identical faces with either neutral or emotionally salient contextual movies, an editing technique referred to as the 'Kuleshov Effect', results in both altered attributions of facial expression and mental-state. Using functional neuroimaging (fMRI), we show that faces paired with emotional movies enhance BOLD responses in the bilateral temporal pole, anterior cingulate cortices, amygdala and bilateral superior temporal sulcus relative to identical faces juxtaposed with neutral movies. An interaction was observed in the right amygdala when subtle happy and fear faces were juxtaposed with positive and negative movies, respectively. An interaction between happy faces and negative context was also observed in bilateral amygdala suggesting that the amygdala may act to prime or tag affective value to faces. A parametric modulation of BOLD signal by attribution ratings indicated a dissociation between ventrolateral and the ventromedial prefrontal cortex for negative and positive contextually evoked attributions, respectively. These prefrontal regions may act to guide appropriate choices across altering contexts. Together, these findings offer a neurobiological basis for contextual framing effects on social attributions.

Fig. 1

(a) Schematic illustration of the paradigm. Participants were presented with 24 randomly allocated contextual epochs in a 3 × 3 factorial design with 8 negative, 8 positive and 8 neutral valance contexts counterbalanced across subjects. Each epoch began with a contextual movie presented for 4 s. A jittered interstimulus interval (ISI) followed, varying between 4 s, 6 s, and 8 s. Following this, a face with a neutral or subtle emotional facial expression was presented for 750 ms. After a short ISI of 650 ms, subjects were required to judge the face at a self-paced rate for emotional expression and mental-state using an orthogonal two-dimensional rating scale. Each epoch lasted up to 2 min, and involved six contextual movies, four or five juxtaposed faces and one or two null events. Each epoch was interleaved with a 17 s rest period. At the end of the scanning session subjects were again asked to rate each face but now with no context provided using the same orthogonal rating scale. (b) Example of neutral and computerized morphs of facial affect. (c) The pseudo-candid photograph manipulation: before each scan, subjects were told that the candid facial expressions were in response to seeing the juxtaposed movie. Subjects were first shown a picture of an actor viewing a movie and a webcam recording expressive facial responses to the movies. A representative picture of the actor's facial response to the movie was shown followed by an edited version on a black background. To protect against the possible confound of repetition effects, subjects were told that no face was shown more than once, although faces look similar due to the subtly of the expressions. To make this plausible, subjects were told that the study was concerned with real-life subtle facial emotions. (d) Mean valance (±s.e.m) and distribution of the IAPS pictures. These pictures were later converted into short movies.

Fig. 2

(a) Mean percentage ratings and mean standard error (±s.e.m) ratings for face expression were significantly influenced by both positive and negative compared with neutral context and post-scan rating of faces. No significant differences were found between neutral faces and post-scan ‘out of context’ ratings. (b) Mental-state ratings showed a similar trend. Statistical parametric maps (from right hemisphere to left hemisphere) and parameter estimate plots illustrating the main effect of faces presented in the (c) negative-neutral contexts and (d) positive-neutral contexts. Faces presented in both negative and positive contexts resulted in increased activity in the STS (negative: 46, −40, −4; Z = 2.88; positive: 56, −22, 2; Z = 3.52), temporal pole (TP) (negative: 52, 2, −38; Z = 4.17; positive: 42,4, −48; Z = 3.62) and ACC (negative: −2, 28, 20; Z = 3.03; positive: 8, 50, 12; Z = 3.74). Amygdala activity was observed for both the negative (34, −2, −18; Z = 2.64) and positive context (22, −2, −26; Z = 2.54) (Table 1).

Fig. 3

Mean percentage ratings and ±s.e.m ratings for (a) each face expression and (b) mental-state ratings in each context. (c) Interaction between fearful faces and negative context and betas for fearful faces presented in the positive (black bars), neutral (dark grey bars) and negative (light grey bars) contexts; (d) Interaction between happy faces and positive contexts and betas for happy faces presented in positive, neutral and negative contexts.

Fig. 4

Correlation between BOLD signal and (e) positive (vmPFC: 0, 44, −6, Z = 3.61) and (f) negative (vlPFC: −38, 46, −6, Z = 3.08) ratings of face expression associated with the neutral faces (Table 3).