Seeing through disguise: Getting to know you with a deep convolutional neural network

Abstract

People use disguise to look unlike themselves (evasion) or to look like someone else (impersonation). Evasion disguise challenges human ability to see an identity across variable images; Impersonation challenges human ability to tell people apart. Personal familiarity with an individual face helps humans to see through disguise. Here we propose a model of familiarity based on high-level visual learning mechanisms that we tested using a deep convolutional neural network (DCNN) trained for face identification. DCNNs generate a face space in which identities and images co-exist in a unified computational framework, that is categorically structured around identity, rather than retinotopy. This allows for simultaneous manipulation of mechanisms that contrast identities and cluster images. In Experiment 1, we measured the DCNN's baseline accuracy (unfamiliar condition) for identification of faces in no disguise and disguise conditions. Disguise affected DCNN performance in much the same way it affects human performance for unfamiliar faces in disguise (cf. Noyes & Jenkins, 2019). In Experiment 2, we simulated familiarity for individual identities by averaging the DCNN-generated representations from multiple images of each identity. Averaging improved DCNN recognition of faces in evasion disguise, but reduced the ability of the DCNN to differentiate identities of similar appearance. In Experiment 3, we implemented a contrast learning technique to simultaneously teach the DCNN appearance variation and identity contrasts between different individuals. This facilitated identification with both evasion and impersonation disguise. Familiar face recognition requires an ability to group images of the same identity together and separate different identities. The deep network provides a high-level visual representation for face recognition that supports both of these mechanisms of face learning simultaneously.

Keywords: Disguise; Face recognition; Machine learning.

Figure 1.

Example of same-identity (top row) and different-identity image pairs (middle and bottom row) in no disguise and disguise conditions. The far-right image in the top row provides and an example of evasion disguise, the far-right image in the middle row provides an example of impersonation similar disguise, and the far-right image in the bottom row is an example of impersonation random disguise.

Figure 2.

Patterns of performance accuracy for the DCNN mirrors that of unfamiliar human participants from Noyes and Jenkins (2019).

Figure 3.

Example of the types of ambient images provided by the models for use in this experiment. The images in Figure 3 are illustrative of the experimental stimuli and depicts author EN who did not appear in the experiments.