Cross-Domain Human Activity Recognition Using Low-Resolution Infrared Sensors

Abstract

This paper investigates the feasibility of cross-domain recognition for human activities captured using low-resolution 8 × 8 infrared sensors in indoor environments. To achieve this, a novel prototype recurrent convolutional network (PRCN) was evaluated using a few-shot learning strategy, classifying up to eleven activity classes in scenarios where one or two individuals engaged in daily tasks. The model was tested on two independent datasets, with real-world measurements. Initially, three different networks were compared as feature extractors within the prototype network. Following this, a cross-domain evaluation was conducted between the real datasets. The results demonstrated the model's effectiveness, showing that it performed well regardless of the diversity of samples in the training dataset.

Keywords: cross-domain; few-shot learning; human activity recognition; long short-term memory networks; low-resolution infrared; prototypes network; recurrent convolutional network.

Figure 1

The model uses data from the support set to generate the prototypes and classify the samples from the query set. The accuracy is measured using the number of matches in the last step. Then, the model updates the network weights of the embedding function in a new iteration. The embedding function corresponds with the PRCN model.

Figure 2

Example of bicubic interpolation from 8 × 8 to 64 × 64 LRIR.

Figure 3

Example of a vectorized spatiotemporal 2D array. The original image was divided into three images with a shape of 20 frames × 64 pixels. Each row corresponds to one flattened 8 × 8 LRIR.

Figure 4

Embedding functions for the three prototypical networks used in this work: (a) PCN (prototypical convolutional network), (b) PRN (prototypical recurrent network), and (c) PRCN (prototypical recurrent convolutional network).

Figure 5

Layouts and environments of two experiment venues.

Figure 6

PCN, PRN, and PRCN confusion matrices for InfraADL Sensor 2 data.

Figure 7

PRCN confusion matrices for Coventry Sensor 2 data with L = 64 samples per class, using (a) a non-pretrained model, (b) a pretrained model with InfraADL sensor 2 data.

Figure 8

PRCN confusion matrices for InfraADL Sensor 2 data according to L (samples per class), using a pretrained model with Coventry Sensor 2 data. These results correspond to Table 7.