Deep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks

Abstract

Photoplethysmography (PPG)-based continuous heart rate monitoring is essential in a number of domains, e.g., for healthcare or fitness applications. Recently, methods based on time-frequency spectra emerged to address the challenges of motion artefact compensation. However, existing approaches are highly parametrised and optimised for specific scenarios of small, public datasets. We address this fragmentation by contributing research into the robustness and generalisation capabilities of PPG-based heart rate estimation approaches. First, we introduce a novel large-scale dataset (called PPG-DaLiA), including a wide range of activities performed under close to real-life conditions. Second, we extend a state-of-the-art algorithm, significantly improving its performance on several datasets. Third, we introduce deep learning to this domain, and investigate various convolutional neural network architectures. Our end-to-end learning approach takes the time-frequency spectra of synchronised PPG- and accelerometer-signals as input, and provides the estimated heart rate as output. Finally, we compare the novel deep learning approach to classical methods, performing evaluation on four public datasets. We show that on large datasets the deep learning model significantly outperforms other methods: The mean absolute error could be reduced by 31 % on the new dataset PPG-DaLiA, and by 21 % on the dataset WESAD.

Keywords: CNN; PPG; dataset; deep learning; evaluation methods; heart rate; time-frequency spectrum.

Figure 1

Time-frequency spectra of PPG-signals, derived from the IEEE datasets. Left plot: IEEE_Training dataset, session $\#8$. Right plot: IEEE_Test dataset, session $\#6$ (subject 5, exercise type 2).

Figure 2

Data collection protocol: example recording of subject S7. X-axis: activities and transient periods performed during the protocol. Y-axis: heart rate, extracted from the ECG signal.

Figure 3

Example ECG-signal snippet from the data recording of subject S1. The two encircled R-peaks were falsely identified by the R-peak detector, and were thus manually removed during heart rate ground truth generation.

Figure 4

IEEE_Training dataset, session S10: comparison of ground truth ($H{R}_{ref}$) and estimated ($H{R}_{est}$) heart rate. Left plot: heart rate estimation with the SpaMa-approach, right plot: heart rate estimation with the SpaMaPlus-approach.

Figure 5

Proposed CNN-architecture with $N_L=1\cdots 8$ convolution-maxpool layers. $N_{tr}$ refers to the number of segments used together for heart rate tracking (see Section 4.1 for details). N depends on $N_L$. Input: $N_{tr}\times N_{ch}\times N_{FFT}$ matrix. Output: heart rate [bpm].

Figure 6

Evaluation results on IEEE_Training (left) and IEEE_Test (right) datasets, investigating the number of convolutional layers and ensemble prediction. LOSO results of the classical methods displayed for reference. $N_L$ refers to the hyperparameter defining the number of convolution-pooling layers, see Section 4.1.

Figure 7

Heart rate estimation on the entire session S7 of the dataset PPG-DaLiA: Ground truth vs. prediction based on the SpaMaPlus approach vs. prediction based on our deep learning model (CNN ensemble). Activity labels are displayed in the lower subplot of the figure.

Figure 8

Heart rate estimation on the cycling part of session S7 of the dataset PPG-DaLiA: Ground truth vs. prediction based on the SpaMaPlus approach vs. prediction based on our deep learning model (CNN ensemble). Activity labels are displayed in the lower subplot of the figure.

Figure 9

Proposed resource-constrained CNN-model. Input: $257\times 4$ matrix (time-frequency spectra of the PPG-signal and three acceleration channels). Output: heart rate [bpm].