A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods

Abstract

The interest in contactless or remote heart rate measurement has been steadily growing in healthcare and sports applications. Contactless methods involve the utilization of a video camera and image processing algorithms. Recently, deep learning methods have been used to improve the performance of conventional contactless methods for heart rate measurement. After providing a review of the related literature, a comparison of the deep learning methods whose codes are publicly available is conducted in this paper. The public domain UBFC dataset is used to compare the performance of these deep learning methods for heart rate measurement. The results obtained show that the deep learning method PhysNet generates the best heart rate measurement outcome among these methods, with a mean absolute error value of 2.57 beats per minute and a mean square error value of 7.56 beats per minute.

Keywords: deep learning; heart rate measurement methods; remote PPG.

Figure 1

PPG signal processing framework.

Figure 2

Illustration of rPPG generation: diffused and specular reflections of ambient illuminance are captured by a camera with the diffused reflection indicating volumetric changes in blood vessels.

Figure 3

rPPG or contactless PPG image processing framework: signal extraction step (ROI detection and tracking), signal estimation step (filtering and dimensionality reduction), and heart rate estimation step (frequency analysis and peak detection).

Figure 4

EVM-CNN modules.

Figure 5

rPPG using SPAD camera.

Figure 6

DeepPhys architecture.

Figure 7

Deep PPG architecture.

Figure 8

HR-CNN modules.

Figure 9

Process of generating synthetic data.

Figure 10

Architecture of STVEN-rPPGNet.

Figure 11

Architecture of iPPG-3D-CNN.

Figure 12

Architecture of PhysNet.

Figure 13

Architecture of Meta-rPPG.

Figure 14

Averaged heart rate measurement of all the subjects in the test set. The vertical axis indicates the heart rate for each method in bpm. Each bar shows the mean and the standard deviation of a method. The first bar from the left indicates the reference.

Figure 15

Heart rate bar charts of all the subjects in the test set for the four compared deep learning methods. Each part of figure (a–j) corresponds to one subject in the test set. The vertical axis indicates the heart rate in bpm. The mean and the standard deviation of each subject are specified in separate bar charts. In each chart, the first bar from the left indicates the reference for a subject.