Surgical face masks impair human face matching performance for familiar and unfamiliar faces

Abstract

In response to the COVID-19 pandemic, many governments around the world now recommend, or require, that their citizens cover the lower half of their face in public. Consequently, many people now wear surgical face masks in public. We investigated whether surgical face masks affected the performance of human observers, and a state-of-the-art face recognition system, on tasks of perceptual face matching. Participants judged whether two simultaneously presented face photographs showed the same person or two different people. We superimposed images of surgical masks over the faces, creating three different mask conditions: control (no masks), mixed (one face wearing a mask), and masked (both faces wearing masks). We found that surgical face masks have a large detrimental effect on human face matching performance, and that the degree of impairment is the same regardless of whether one or both faces in each pair are masked. Surprisingly, this impairment is similar in size for both familiar and unfamiliar faces. When matching masked faces, human observers are biased to reject unfamiliar faces as "mismatches" and to accept familiar faces as "matches". Finally, the face recognition system showed very high classification accuracy for control and masked stimuli, even though it had not been trained to recognise masked faces. However, accuracy fell markedly when one face was masked and the other was not. Our findings demonstrate that surgical face masks impair the ability of humans, and naïve face recognition systems, to perform perceptual face matching tasks. Identification decisions for masked faces should be treated with caution.

Keywords: Deep neural network; Face recognition; Familiarity; Identity verification; Signal detection theory.

Fig. 1

Examples of the control, mixed and masked conditions for match and mismatch trials of the GFMT [ reproduced and adapted with permission from the copyright holder]

Fig. 2

a Sensitivity (d′) and b response bias (criterion c) on the GFMT and the SFFMT (plotted separately for familiar and unfamiliar faces). Positive criterion c values indicate a conservative response bias (inclined to say ‘mismatch’), while negative values indicate a liberal bias. All error bars show the standard error of the mean (SEM)

Fig. 3

Classification accuracy of the DNN, shown as the percentage correct of the 20 trials in each condition, plotted separately for the match and mismatch trials of the GFMT and the SFFMT

Fig. 4

Similarity ratings from the DNN, plotted separately for the match and mismatch trials of the GFMT and the SFFMT. Error bars show SEM