A Self-Adaptive Gallery Construction Method for Open-World Person Re-Identification

Abstract

Person re-identification, or simply re-id, is the task of identifying again a person who has been seen in the past by a perception system. Multiple robotic applications, such as tracking or navigate-and-seek, use re-identification systems to perform their tasks. To solve the re-id problem, a common practice consists in using a gallery with relevant information about the people already observed. The construction of this gallery is a costly process, typically performed offline and only once because of the problems associated with labeling and storing new data as they arrive in the system. The resulting galleries from this process are static and do not acquire new knowledge from the scene, which is a limitation of the current re-id systems to work for open-world applications. Different from previous work, we overcome this limitation by presenting an unsupervised approach to automatically identify new people and incrementally build a gallery for open-world re-id that adapts prior knowledge with new information on a continuous basis. Our approach performs a comparison between the current person models and new unlabeled data to dynamically expand the gallery with new identities. We process the incoming information to maintain a small representative model of each person by exploiting concepts of information theory. The uncertainty and diversity of the new samples are analyzed to define which ones should be incorporated into the gallery. Experimental evaluation in challenging benchmarks includes an ablation study of the proposed framework, the assessment of different data selection algorithms that demonstrate the benefits of our approach, and a comparative analysis of the obtained results with other unsupervised and semi-supervised re-id methods.

Keywords: incremental clustering; open-world recognition; person recognition.

Figure 1

Simplified comparison between a large static gallery, traditionally used, and our small self-adaptive gallery. Both have a set of images representing each identity (ID0, ID1, …), i.e., each person. The traditional gallery is the same for every person query that arrives at different times ($t_i$, $t_f$). However, because the adaptive gallery is being built and updated as new data arrives, we can appreciate a more comprehensive gallery for later times ($t_i<t_n<t_f$).

Figure 2

Self-adaptive gallery construction method overview. The person bounding box undergoes a pre-processing where the sample features are obtained with existing deep neural network encoders. Then, the proposed method analyzes the features obtained to decide which ones are used to adapt and evolve the gallery with the new information.

Figure 3

Gallery optimization. Upper area: example simplified where $C_1$ exceeds the memory budget and the gallery optimization selects the feature with the maximum cost to be dropped, $x_4$. Lower area: visual sample of two appearance models from similar identities that are correctly separated in the DukeMTMC-VideoReID dataset. The yellow edge corresponds to identity 86 and the orange edge to identity 194, both ground truth identities.

Figure 4

Parameter evaluation in the gallery construction process using the MARS dataset: (a) effect of the weight assigned to uncertainty and diversity ($\gamma$); (b) influence of the expansion threshold ($\tau$); (c) effect of the minimum size to create a class and the memory budget ($l/m$).

Figure 5

Evolution over time of the metrics with the final parameters set in the gallery construction process using the MARS dataset .

Figure 6

Data selection method comparison with the MARS dataset. We analyze (a) the number of classes created (x-axis) per GT-ID in the dataset (y-axis) showing the number of GT-ID with more than one class associated or those GT-ID that have not been correctly found, i.e., 0 classes associated; (b) gallery structure metrics: $F1$, precision, and recall; (c) sample classification $F1$ analyzing the influence of varying K; (d) class precision.

Figure 7

Visualization of the evolution of appearance models in the gallery. Each row corresponds to gallery samples at a certain time. The different colors represent the time stamp of the samples included in the gallery (best viewed in color).