Improving face identification with specialist teams

Abstract

People vary in their ability to identify faces, and this variability is relatively stable across repeated testing. This suggests that recruiting high performers can improve identity verification accuracy in applied settings. Here, we report the first systematic study to evaluate real-world benefits of selecting high performers based on performance in standardized face identification tests. We simulated a recruitment process for a specialist team tasked with detecting fraudulent passport applications. University students (n = 114) completed a battery of screening tests followed by a real-world face identification task that is performed routinely when issuing identity documents. Consistent with previous work, individual differences in the real-world task were relatively stable across repeated tests taken 1 week apart (r = 0.6), and accuracy scores on screening tests and the real-world task were moderately correlated. Nevertheless, performance gains achieved by selecting groups based on screening tests were surprisingly small, leading to a 7% improvement in accuracy. Statistically aggregating decisions across individuals-using a 'wisdom of crowds' approach-led to more substantial gains than selection alone. Finally, controlling for individual accuracy of team members, the performance of a team in one test predicted their performance in a subsequent test, suggesting that a 'good team' is not only defined by the individual accuracy of team members. Overall, these results underline the need to use a combination of approaches to improve face identification performance in professional settings.

Keywords: Face recognition; Identity verification; Individual differences; Personnel selection; Super-recognizers; Unfamiliar face matching.

Fig. 1

An illustration of the trial design in the Face Recognition Candidate List Review Task (FR-Task). This task simulated the workflow of passport officers using face recognition software to screen for identity fraud in passport applications. On each trial, participants were asked to review four faces that have been flagged by the software as potential matches to the ‘applicant’. Participants made sequential pairwise similarity ratings by comparing the four faces to the applicant (left), before reviewing and confirming these ratings in a gallery review screen (right). The images shown are representative of the images presented to participants, but for privacy reasons we are unable to reproduce the passport images used in the study

Fig. 2

Accuracy distributions for face identification tests. Means are shown in the solid black lines and medians in the red dashed lines. The width of each ‘violin’ represents the expected probability density of that score based on the data and assuming a Gaussian distribution. Individual data points are shown as grey dots. See main text for details. CFMT Cambridge face memory test, FR-Task Face Recognition Candidate List Review Task, GFMT Glasgow face matching test

Fig. 3

Results of Analysis 2, showing session-two Face Recognition Candidate List Review Task (FR-Task) performance in groups selected using session one screening tests. Continuous black lines show the average of all participants (n = 114). Violin plots are presented in three columns pertaining to three selection thresholds (above mean, above one standard deviation (SD), top 10 performers), and show session-two FR Test performance for groups performing above the threshold in each screening measure. Within each violin, black lines represent group mean, red lines are the median, and data points are individual participants. Because only one participant was in the Top 10 performers for face identification accuracy on all screening tests, an orange point shows their performance. CFMT Cambridge face memory test, GFMT Glasgow face matching test, PI-20 20-item prosopagnosia index, ROC receiver operating characteristic

Fig. 4

Results of Analysis 3 showing average team performance on FR-Task in session two as a function of the number of individual responses aggregated. The solid line shows teams selected from the whole pool of participants and the dashed line shows teams selected from individuals achieving an average score of above one standard deviation on the session-one face tests. For comparison, crowd size of 1 is included, representing average performance of individuals. Details are provided in the main text. AUC area under the curve

Fig. 5

Correlation between FR-Task accuracy of teams in sessions one and two (Analysis 4). Individual data points represent the accuracy achieved by averaging the responses of teams containing three individuals. See text for details

Fig. 6

Regression analysis examining the factors underpinning stability in team accuracy across sessions one and two (Analysis 4). a Scatterplots showing team performance resulting from response aggregation (y axis) as a function of average accuracy of individual team members (x axis). Session one data are on the left scatterplot and session two on the right. b Scatterplot of residuals from the expected team accuracy based on linear regression in (a) for session one as a function of session two. Details of this analysis are provided in the main text