Automated video-based heart rate tracking for the anesthetized and behaving monkey

Abstract

Heart rate (HR) is extremely valuable in the study of complex behaviours and their physiological correlates in non-human primates. However, collecting this information is often challenging, involving either invasive implants or tedious behavioural training. In the present study, we implement a Eulerian video magnification (EVM) heart tracking method in the macaque monkey combined with wavelet transform. This is based on a measure of image to image fluctuations in skin reflectance due to changes in blood influx. We show a strong temporal coherence and amplitude match between EVM-based heart tracking and ground truth ECG, from both color (RGB) and infrared (IR) videos, in anesthetized macaques, to a level comparable to what can be achieved in humans. We further show that this method allows to identify consistent HR changes following the presentation of conspecific emotional voices or faces. EVM is used to extract HR in humans but has never been applied to non-human primates. Video photoplethysmography allows to extract awake macaques HR from RGB videos. In contrast, our method allows to extract awake macaques HR from both RGB and IR videos and is particularly resilient to the head motion that can be observed in awake behaving monkeys. Overall, we believe that this method can be generalized as a tool to track HR of the awake behaving monkey, for ethological, behavioural, neuroscience or welfare purposes.

Figure 1

(A) HR video extraction pipeline. A Region of interest (ROI) is defined on the video input, ideally placed in a hairless skin region, such as the face. The video is cropped around this ROI selection and serves as input to the Eulerian video magnification (EVM) algorithm (see text for specifications). The result is a video in which luminosity changes due to HR are enhanced. A first HR approximation is extracted from this video (as described in B). In order to obtain a more precise HR estimate, this approximated HR is used as a parameter for a second EVM processing round. (B) HR approximation. Average time series are extracted from the ROI pixels of highest luminosity, for each color channel, and their frequency power profile is extracted using a wavelet transform. Peak frequency is taken as the estimate of HR.

Figure 2

Comparing ECG and EVM HR estimates. (A) Raw ECG exemplar sample. (B) ECG inter-peak interval estimate (gray) and pulse rate running average (black) in time. (C) Time frequency spectrogram of EVM processed video data. Highest power frequency band (yellow) corresponds to the HR estimate. (D) EVM peak frequency estimate (gray) and corresponding pulse rate running average (light gray) in time. (E) Overlay of ECG (black) and EVM (light gray) based running average pulse rates estimates in time. (F) Coherence between ECG and EVM HR estimates.

Figure 3

HR estimation in humans (panels 1 & 2) and monkey (panels 3 & 4) based on RGB (panels 1 & 3) and IR (panels 2 & 4). (A) ECG HR ground truth. (B) EVM HR estimate. (C) Coherence between ECG and EVM HR estimates.

Figure 4

Effect of ROI size and localization on ECG-EVM pulse rate coherence. (A) Reference monkey face on which is superimposed ECG-EVM coherence maximum power based on a 10 × 10 pixel matrix covering the entire face (B) or smaller pixels covering the eyes and muzzle (C). Note that B and C share the same color scale.

Figure 5

EVM HR estimate modulation (mean + /− s.e.) by (A) monkey screams (Wilcoxon test comparing pre-stimulus [− 400 − 100 ms] and post-stimulus [100 400] epochs, p = 0.03) and (B) monkey aggressive faces (p = 0.001).