Dissociating neural signatures of mental state retrodiction and classification based on facial expressions

Abstract

Posed facial expressions of actors have often been used as stimuli to induce mental state inferences, in order to investigate 'Theory of Mind' processes. However, such stimuli make it difficult to determine whether perceivers are using a basic or more elaborated mentalizing strategy. The current study used as stimuli covert recordings of target individuals who viewed various emotional expressions, which caused them to spontaneously mimic these expressions. Perceivers subsequently judged these subtle emotional expressions of the targets: in one condition ('classification') participants were instructed to classify the target's expression (i.e. match it to a sample) and in another condition ('retrodicting') participants were instructed to retrodict (i.e. infer which emotional expression the target was viewing). When instructed to classify, participants showed more prevalent activations in event-related brain potentials (ERPs) at earlier and mid-latency ERP components N170, P200 and P300-600. By contrast, when instructed to retrodict participants showed enhanced late frontal and fronto-temporal ERPs (N800-1000), with more sustained activity over the right than the left hemisphere. These findings reveal different cortical processes involved when retrodicting about a facial expression compared to merely classifying it, despite comparable performance on the behavioral task.

Fig. 1

A schematic diagram illustrating the two stages of the experiment. The target looked at the photograph of an expression while his/her own expression was covertly recorded (‘Target Phase’). The perceiver then watched the recording of the target and guessed what photograph the target was looking at (‘Perceiver Phase’); (b) example of peak expressions used as stimuli—(from left to right) happiness, surprise, disgust and anger; (c) experimental paradigm for both classifying and retrodicting perceivers in the ‘Perceiver Phase’. Participants saw a cue word for 500 ms, followed by a fixation cross for 500 ms. Then, the target’s expression was displayed for 1000 ms. After that, participants had to make a response on the judgment scale (depicting Ekman-type expressions of happy, surprised, anger, disgusted and neutral) which was displayed for 5000 ms. Finally, a 500 ms fixation cross appeared.

Fig. 2

Mean unbiased hit rates for all viewed target peak expressions (i.e. happy, surprise, angry and disgust) for both perceiver groups. Error bars represent the SEM. The dotted line represents chance level performance.

Fig. 3

N170 early occipito-temporal ERPs across hemispheres as a function of target expression for the classification group only. Note the larger amplitudes over the right compared to the left hemisphere. In the retrodiction group, no hemisphere by target expression interaction was found.

Fig. 4

ERP waveforms for the retrodiction and classification groups across all 64 channels, averaged across all other experimental conditions.

Fig. 5

Voltage maps depicting the topographical differences between groups, for all time segments analyzed, and collapsed across the other experimental conditions. The top row shows a top view map using a 110-degree equidistant projection. Middle and bottom rows show a left and right view, respectively, both using a 90-degree equidistant projection. All maps were obtained using spherical spline interpolation. Electrode positions are also shown. Negativity is blue.

Fig. 6

Late frontal and fronto-temporal ERP asymmetries (800–1000 ms) for both groups. Left half of the figure: note that ERP negativity over the left hemisphere for the retrodiction group returns toward baseline whereas a distinct sustained activity over the right hemisphere can be observed. For the classification group, negativity over the right hemisphere returns toward baseline whereas a sustained negativity is seen over the left hemisphere. Right half of the figure: the same data plotted with ERPs from homologue electrodes over both hemispheres superimposed, such that hemispheric asymmetries become more readily visible. A vertical line is plotted at 800 ms.

Fig. 7

Voltage maps for mean amplitudes between 800 and 1000 ms, depicting late frontal hemispheric asymmetries for the retrodiction (left) and classification (right) groups. Top view maps use a 110-degree equidistant projection; left and right view maps both use a 90-degree equidistant projection. All maps were obtained using spherical spline interpolation. Electrode positions are also shown. Negativity is blue.