Amplitude-selective filtering for remote-PPG

Abstract

Biometric signatures of remote photoplethysmography (rPPG), including the pulse-induced characteristic color absorptions and pulse frequency range, have been used to design robust algorithms for extracting the pulse-signal from a video. In this paper, we look into a new biometric signature, i.e., the relative pulsatile amplitude, and use it to design a very effective yet computationally low-cost filtering method for rPPG, namely "amplitude-selective filtering" (ASF). Based on the observation that the human relative pulsatile amplitude varies in a specific lower range as a function of RGB channels, our basic idea is using the spectral amplitude of, e.g., the R-channel, to select the RGB frequency components inside the assumed pulsatile amplitude-range for pulse extraction. Similar to band-pass filtering (BPF), the proposed ASF can be applied to a broad range of rPPG algorithms to pre-process the RGB-signals before extracting the pulse. The benchmark in challenging fitness use-cases shows that applying ASF (ASF+BPF) as a pre-processing step brings significant and consistent improvements to all multi-channel pulse extraction methods. It improves different (multi-wavelength) rPPG algorithms to the extent where quality differences between the individual approaches almost disappear. The novelty of the proposed method is its simplicity and effectiveness in providing a solution for the extremely challenging application of rPPG to a fitness setting. The proposed method is easy to understand, simple to implement, and low-cost in running. It is the first time that the physiological property of pulsatile amplitude is used as a biometric signature for generic signal filtering.

Keywords: (170.3880) Medical and biological imaging; (280.0280) Remote sensing and sensors.

Fig. 1

Illustration of the amplitude-selective filtering. The essence of this filter is in the step of selecting the RGB frequency components within the pulsatile amplitude-range for pulse extraction, thus removing large components due to motion.

Fig. 2

(a) The statistically measured relative pulsatile amplitude as a function of RGB channels. (b) The continuous relative pulsatile traces exemplified in extreme lighting conditions and the optical filter responses of the used RGB camera.

Fig. 3

Experimental setup and video snapshots.

Fig. 4

Illustration of the two quality metrics (SNR and success-rate) used for evaluating the rPPG performance. In the SNR metric, the frequency components of pulse (green) and noise (red) are defined by the ECG-reference. In the success-rate metric, the inlier estimates (green) and outlier estimates (red) are defined by a tolerance (dashed black line) w.r.t. the ECG-reference.

Fig. 5

The SNR comparison between eight rPPG algorithms as a function of pre-processing. Different panels show the results obtained by using different filters (e.g., None, BPF, ASF and ASF+BPF) in the pre-processing, where the median values are indicated by red bars, the quartile range by blue boxes, the full range by whiskers, disregarding the outliers (red crosses).

Fig. 6

The success-rate curves (and corresponding AUC) obtained by eight rPPG algorithms over 23 benchmark videos by using different filters in the pre-processing. Each panel shows the contribution of three filters (i.e., BPF, ASF and ASF+BPF) to a particular rPPG algorithm, where different colors denote the AUC for different filters and the percentage numbers exemplify their success-rate at T = 3, i.e., allowing 3 bpm difference with the ECG-reference.

Fig. 7

Spectrograms obtained by eight rPPG algorithms on a fitness video by using different filters in the pre-processing. From top to bottom: baseline without pre-processing (None), Band-Pass Filter (BPF), Amplitude-Selective Filter (ASF), and the combination of ASF and BPF (ASF+BPF).

Fig. 8

Spectrograms obtained by eight rPPG algorithms on a fitness video by using ASF in the pre-processing (Pre-ASF) or the post-processing (Post-ASF). From top to bottom: baseline without pre-/post- processing (None), Pre-ASF, and Post-ASF.

Fig. 9

The 3D mesh shows different SNR (first row) and AUC of success-rate (second row) when varying the parameters of ASF for eight rPPG algorithms. The two varied parameters are: the sliding window length L (first column) and the maximum amplitude threshold a_max (second column). When changing the investigated parameter, the other one remains constant. The red/blue color represents the high/low values for SNR and AUC of success-rate.