Zygomaticus activation through facial neuromuscular electrical stimulation (fNMES) induces happiness perception in ambiguous facial expressions and affects neural correlates of face processing

Abstract

The role of facial feedback in facial emotion recognition remains controversial, partly due to limitations of the existing methods to manipulate the activation of facial muscles, such as voluntary posing of facial expressions or holding a pen in the mouth. These procedures are indeed limited in their control over which muscles are (de)activated when and to what degree. To overcome these limitations and investigate in a more controlled way if facial emotion recognition is modulated by one's facial muscle activity, we used computer-controlled facial neuromuscular electrical stimulation (fNMES). In a pre-registered EEG experiment, ambiguous facial expressions were categorised as happy or sad by 47 participants. In half of the trials, weak smiling was induced through fNMES delivered to the bilateral Zygomaticus Major muscle for 500 ms. The likelihood of categorising ambiguous facial expressions as happy was significantly increased with fNMES, as shown with frequentist and Bayesian linear mixed models. Further, fNMES resulted in a reduction of P1, N170 and LPP amplitudes. These findings suggest that fNMES-induced facial feedback can bias facial emotion recognition and modulate the neural correlates of face processing. We conclude that fNMES has potential as a tool for studying the effects of facial feedback.

Keywords: embodiment; event-related potentials; fNMES; face perception; facial feedback.

Fig. 1.

(A) Example of the stimuli used: here is a female avatar, with emotional expressions changing from 30% sadness to 30% happiness in steps of 10%. (B) In each trial, participants viewed a fixation cross for 1250 ms, followed by an avatar face for 500 ms. In the fNMES on condition, electrical stimulation was delivered to the ZM muscle to induce a weak smile. In the off condition, there was no electrical stimulation and participants maintained a neutral expression. Thereafter, participants viewed a scrambled face for a jittered time interval of 750–950 ms, and finally, participants had up to 3000 ms to respond via button press to indicate the perceived emotion of the non-scrambled facial expression (happy or sad).

Fig. 2.

Baseline-corrected results from the OpenFace analysis of video recordings (500–2000 ms) of participants’ faces, based on the FACS (Ekman et al., 2002). The activation of four AUs (AU6, AU12, AU15 and AU4), averaged across all trials where algorithm confidence was > 95%, is shown for trials with (orange) and without (blue) fNMES. Notice how fNMES delivery (period indicated by the shaded area) resulted in a 40% activation of AU12, which corresponds to the ZM muscle, followed by a faint activation of AU6 and a relaxation of AU15 (an antagonistic muscle). Importantly, fNMES delivery did not result in AU4 activation (reflecting frowning), which would have been suggestive of a pain response or negative emotion induction. The shaded grey region on the line represents the SE.

Fig. 3.

The predicted values for the main effect of fNMES on happy responses to facial stimuli varying from 30% sad to 30% happy. Panel (A) shows the percentage of happy responses across Emotion and fNMES, using the marginal means of the model. Individual dots display participants (jittered to improve visibility), and the dark point reflects the mean with SE bars (SE). Panel (B) displays the mean difference (and SE) in the percentage of happy responses between fNMES conditions (on minus off) across emotion levels. The shaded points represent participant means.

Fig. 4.

Prior and posterior predictions for the group-level effects of the Bayesian GLMM. Panel A represents the fixed effects, showing the percentage of choice happiness based on the percentage of emotion in the stimulus, ranging from sad to happy. The points are colour-coded by fNMES, and the shaded ribbon represents the uncertainty of the estimates. Panel B visualises the posterior predictions, with the ribbon showing uncertainty in the estimate, and points display individual participants (jittered for visibility).

Fig. 5.

Panel A shows the time series of the ERP for the P1 component (80–140 ms), panel B shows the time series of the ERP for the N170 component (130–190 ms) and panel C shows the time series of the ERP for the LPP component (450–650 ms). The shaded grey area in each panel indicates the time region used for statistical analysis. Overall, after subtracting the fNMES-only trials, we observed a reduction in amplitude for all three components during fNMES on relative to fNMES off. Panel D shows the topography of each ERP component for its respective time, shaded in grey. Panel E shows the main effect of emotion on LPP amplitude, and error bars show the standard error. * < 0.05.