Agreement between two photoplethysmography-based wearable devices for monitoring heart rate during different physical activity situations: a new analysis methodology

Abstract

Wearables are being increasingly used to monitor heart rate (HR). However, their usefulness for analyzing continuous HR in research or at clinical level is questionable. The aim of this study is to analyze the level of agreement between different wearables in the measurement of HR based on photoplethysmography, according to different body positions and physical activity levels, and compared to a gold-standard ECG. The proposed method measures agreement among several time scales since different wearables obtain HR at different sampling rates. Eighteen university students (10 men, 8 women; 22 ± 2.45 years old) participated in a laboratory study. Participants simultaneously wore an Apple Watch and a Polar Vantage watch. ECG was measured using a BIOPAC system. HR was recorded continuously and simultaneously by the three devices, for consecutive 5-min periods in 4 different situations: lying supine, sitting, standing and walking at 4 km/h on a treadmill. HR estimations were obtained with the maximum precision offered by the software of each device and compared by averaging in several time scales, since the wearables obtained HR at different sampling rates, although results are more detailed for 5 s and 30 s epochs. Bland-Altman (B-A) plots show that there is no noticeable difference between data from the ECG and any of the smartwatches while participants were lying down. In this position, the bias is low when averaging in both 5 s and 30 s. Differently, B-A plots show that there are differences when the situation involves some level of physical activity, especially for shorter epochs. That is, the discrepancy between devices and the ECG was greater when walking on the treadmill and during short time scales. The device showing the biggest discrepancy was the Polar Watch, and the one with the best results was the Apple Watch. We conclude that photoplethysmography-based wearable devices are suitable for monitoring HR averages at regular intervals, especially at rest, but their feasibility is debatable for a continuous analysis of HR for research or clinical purposes, especially when involving some level of physical activity. An important contribution of this work is a new methodology to synchronize and measure the agreement against a gold standard of two or more devices measuring HR at different and not necessarily even paces.

Figure 1

Signal processing stages. The Sync block synchronizes the time stamps of the point heart rate estimates for the alternative measurement methods while the Aver block provides an average of point estimates for each method at the same reference time stamp. See text for further details.

Figure 2

Differences of HR mean values between GS and the other devices. The mean value of the differences and their 95% confidence interval are represented as well as the significance of Bonferroni Contrast Test *p < .05; **p < .01; ***p < .001 and the η_p² measuring the effect size.

Figure 3

Change in the Limits of agreement (LoA) with 2.5% and 97.5% percentiles of the Bland–Altman plots with respect to the averaging time (from 5 to 60 s) for the four activities. Red and black lines correspond to the PV and AW devices respectively.

Figure 4

Median differences of the Bland–Altman for the AW and PV devices for 5 s and 30 s averaging time and for the four activities. Red bars are for AW device and 5 s averaging time, dark blue are for PV device and 5 s, brown are for AW device and 30 s and light blue are for PV device and 30 s. Wilcoxon Rank Sum Test results are also shown comparing the median values of differences for both devices. Significant differences: ‡ p < .001.

Figure 5

Standard deviation of the differences in the Bland–Altman for the AW and PV devices for 5 s and 30 s averaging time and for the four activities. Red bars are for AW device and 5 s averaging time, dark blue are for PV device and 5 s, brown are for AW device and 30 s and light blue are for PV device and 30 s. Ansari-Bradley Test results are also shown comparing the spread of the differences for both devices. Significant differences: ‡ p < .001; † p < .05.