Application of artificial intelligence to eyewitness identification

Abstract

Artificial intelligence is already all around us, and its usage will only increase. Knowing its capabilities is critical. A facial recognition system (FRS) is a tool for law enforcement during suspect searches and when presenting photos to eyewitnesses for identification. However, there are no comparisons between eyewitness and FRS accuracy using video, so it is unknown whether FRS face matches are more accurate than eyewitness memory when identifying a perpetrator. Ours is the first application of artificial intelligence to an eyewitness experience, using a comparative psychology approach. As a first step to test system accuracy relative to eyewitness accuracy, participants and an open-source FRS (FaceNet) attempted perpetrator identification/match from lineup photos (target-present, target-absent) after exposure to real crime videos with varied clarity and perpetrator race. FRS used video probe images of each perpetrator to achieve similarity ratings for each corresponding lineup member. Using receiver operating characteristic analysis to measure discriminability, FRS performance was superior to eyewitness performance, regardless of video clarity or perpetrator race. Video clarity impacted participant performance, with the unclear videos yielding lower performance than the clear videos. Using confidence-accuracy characteristic analysis to measure reliability (i.e., the likelihood the identified suspect is the actual perpetrator), when the FRS identified faces with the highest similarity values, they were accurate. The results suggest FaceNet, or similarly performing systems, may supplement eyewitness memory for suspect searches and subsequent lineup construction and knowing the system's strengths and weaknesses is critical.

Keywords: Estimator variables; Eyewitness accuracy; Face-recognition software; Legal implications; Race effects; Viewing context.

Fig. 1

Screenshots from the clear and unclear videos. There were 3 clear and 3 unclear videos (one for each perpetrator race)

Fig. 2

Diagram of the procedure for the human participants. Participants viewed all six videos in a random order. After viewing each video, participants completed the same sequence of events (distraction task one, identification task of a target-present or target-absent lineup, confidence rating, distractor task two). Prior to each identification task, participants read standard instructions

Fig. 3

Receiver operating characteristic (ROC) curves for the face recognition system (FRS) and participants. The shaded regions represent the partial area under the curve (pAUC) regions for each curve, using the cut-off of the overall false ID rate of the FRS. The error bars are 68% confidence intervals based on 200 bootstraps. The dashed line represents chance performance. Point sizes reflect relative frequencies of responses

Fig. 4

Receiver operating characteristic (ROC) curves of the FRS and participants for the clear and unclear conditions. The shaded regions represent the partial area under the curve (pAUC) regions for each condition, using the cut-off of the overall false ID rate of the FRS clear condition. The error bars are 68% confidence intervals based on 200 bootstraps. The black dashed line represents chance performance. Point sizes reflect relative frequencies of responses

Fig. 5

Receiver operating characteristic curves for the race conditions. The shaded regions represent the pAUC per condition, using the cut-off of the overall false ID rate of the FRS White condition. The error bars are 68% confidence intervals based on 200 bootstraps. The dashed line represents chance performance. Point sizes reflect relative frequencies of responses

Fig. 6

Confidence accuracy characteristic plots collapsed across conditions for the FRS (left panel) and participants (right panel). The error bars are 68% confidence intervals based on 200 bootstraps. The black dashed line represents chance performance. Point sizes reflect relative frequencies of responses. The faces with the strongest FRS similarity values were 100% accurate (< 1.1)

Fig. 7

Histograms of the Euclidean similarity values for guilty suspect, innocent suspect, and filler distributions for each video. Lower values indicate higher similarities. Values on the y-axis vary as a result of the different number of frames in each video

Fig. 8

Confidence accuracy characteristic plots collapsed across race for the FRS (left panel) and participants (right panel). The error bars are 68% confidence intervals based on 200 bootstraps. The black dashed line represents chance performance. Point sizes reflect relative frequencies of responses

Fig. 9

Confidence accuracy characteristic plots collapsed across clarity for the FRS (left panel) and participants (right panel). The error bars are 68% confidence intervals based on 200 bootstraps. The black dashed line represents chance performance. Point sizes reflect relative frequencies of responses. The faces with the strongest FRS similarity values were 100% accurate (< 1.1)