Drift-diffusion modeling reveals that masked faces are preconceived as unfriendly

Abstract

During the COVID-19 pandemic, the use of face masks has become a daily routine. Studies have shown that face masks increase the ambiguity of facial expressions which not only affects (the development of) emotion recognition, but also interferes with social interaction and judgement. To disambiguate facial expressions, we rely on perceptual (stimulus-driven) as well as preconceptual (top-down) processes. However, it is unknown which of these two mechanisms accounts for the misinterpretation of masked expressions. To investigate this, we asked participants (N = 136) to decide whether ambiguous (morphed) facial expressions, with or without a mask, were perceived as friendly or unfriendly. To test for the independent effects of perceptual and preconceptual biases we fitted a drift-diffusion model (DDM) to the behavioral data of each participant. Results show that face masks induce a clear loss of information leading to a slight perceptual bias towards friendly choices, but also a clear preconceptual bias towards unfriendly choices for masked faces. These results suggest that, although face masks can increase the perceptual friendliness of faces, people have the prior preconception to interpret masked faces as unfriendly.

Figure 1

Ambiguous facial stimuli and model predictions. (A) Average non-binary happy and angry faces were created by morphing the (average) male and female AKDEF faces. Next, emotionally ambiguous faces were created by morphing the happy face towards the angry face in 41 incremental steps of 2.5% each. Six faces were used, each with a different angry/happy ratio. Stimulus emotion was defined by the difference in the terms of the ratio (Angry—Happy). (B) The DDM (Drift Diffusion Model) represents decisions as an accumulation of noisy sensory evidence over time (drift rate v), which starts at starting point (z) and ends at one of the decision thresholds (a or -a). These decision thresholds are collapsing, meaning that they get closer together as the deadline for making a decision approaches, allowing to model the increasing sense of urgency that individuals experience as they approach a decision deadline. Non-decision time (Ter) is the time for processes other than the decision process, such as sensory encoding and execution of the response (grey boxes). (C) Perceptual bias is driven by a shift in the drift-rate criterion (vc), which determines the point at which the perceptual evidence of the facial expression is classified as either unfriendly or friendly. Due to the ‘unfriendliness’ of the mask, the drift-rate criterion will shift in favor of the unfriendly alternative, resulting in a biased accumulation process towards the unfriendly (v₀ + v_mask) and away from the friendly alternative (v₀ − v_mask). (D) Preconceptual bias is driven by a shift in starting point (z), reflecting asymmetric distances to the decision thresholds. Due to an initial unfriendly preconception about the mask, the participant expects that the unfriendly alternative will be the correct one, resulting in a starting-point closer to the unfriendly decision threshold (z₀ + z_mask). In contrast to a shift in the drift-rate criterion, where the stimulus is evaluated differently (1C), a shift in the starting-point does not affect the evaluation process, but rather affects how much evidence is needed for each response. Gray and red dots represent unbiased and biased responses respectively.

Figure 2

Descriptive data. (A) Psychometric functions of the pooled data across participants for masked (blue) and unmasked (red) ambiguous facial expressions. The proportion of unfriendly choices is plotted as a function of stimulus emotion. (B) Group averages of median response times (in seconds) as a function of stimulus emotion, for masked (blue) and unmasked (red) ambiguous facial expressions.

Figure 3

Effects of facial masks on the amount of available perceptual information (v-slope) and bias parameters (vc and z). (A) The presence of facial masks affects the relationship between the stimulus emotion and the drift-rate, resulting in a less pronounced increase in evidence accumulation with each increase in stimulus emotion (v-slope_mask). (B) Effect of facial masks on drift-rate criterion (left: vc_mask) and starting point (right: z_mask) relative to unmasked facial expressions. Values > 0 indicate a bias towards the unfriendly alternative, while values < 0 indicate a bias towards the friendly alternative.

Figure 4

Goodness of fit of the full_vc,z model. The graph shows empirical (x-axis) and predicted (y-axis) data for unmasked (top row) and masked (bottom row) facial stimuli. Predicted data was generated for each participant separately using the individual full_vc,z model parameters. Choice data was plotted together with the quantiles (5th, 10th, 30th, 50th, 70th, 90th) of the RT distributions using RTs for unfriendly (black) and friendly (white) responses on happy and angry facial expressions (collapsed over intensity levels). The plot shows that the predicted values of choice data and RT quantiles are close to the empirical values for most datapoints, suggesting that the model fit the data reasonably well.