A Validation of Six Wearable Devices for Estimating Sleep, Heart Rate and Heart Rate Variability in Healthy Adults

Abstract

The primary aim of this study was to examine the validity of six commonly used wearable devices, i.e., Apple Watch S6, Garmin Forerunner 245 Music, Polar Vantage V, Oura Ring Generation 2, WHOOP 3.0 and Somfit, for assessing sleep. The secondary aim was to examine the validity of the six devices for assessing heart rate and heart rate variability during, or just prior to, night-time sleep. Fifty-three adults (26 F, 27 M, aged 25.4 ± 5.9 years) spent a single night in a sleep laboratory with 9 h in bed (23:00-08:00 h). Participants were fitted with all six wearable devices-and with polysomnography and electrocardiography for gold-standard assessment of sleep and heart rate, respectively. Compared with polysomnography, agreement (and Cohen's kappa) for two-state categorisation of sleep periods (as sleep or wake) was 88% (κ = 0.30) for Apple Watch; 89% (κ = 0.35) for Garmin; 87% (κ = 0.44) for Polar; 89% (κ = 0.51) for Oura; 86% (κ = 0.44) for WHOOP and 87% (κ = 0.48) for Somfit. Compared with polysomnography, agreement (and Cohen's kappa) for multi-state categorisation of sleep periods (as a specific sleep stage or wake) was 53% (κ = 0.20) for Apple Watch; 50% (κ = 0.25) for Garmin; 51% (κ = 0.28) for Polar; 61% (κ = 0.43) for Oura; 60% (κ = 0.44) for WHOOP and 65% (κ = 0.52) for Somfit. Analyses regarding the two-state categorisation of sleep indicate that all six devices are valid for the field-based assessment of the timing and duration of sleep. However, analyses regarding the multi-state categorisation of sleep indicate that all six devices require improvement for the assessment of specific sleep stages. As the use of wearable devices that are valid for the assessment of sleep increases in the general community, so too does the potential to answer research questions that were previously impractical or impossible to address-in some way, we could consider that the whole world is becoming a sleep laboratory.

Keywords: autonomic modulation; autonomic nervous system; cardiovascular health; consumer sleep technology; photoplethysmography; sleep quality; sleep quantity; sleep staging; wearable sleep monitor.

Figure 1

Bland–Altman plots for device–derived and PSG–derived measures of total sleep time. The y-axes represent the difference between the measures—positive values indicate devices overestimate relative to PSG, and negative values indicate devices underestimate relative to PSG. Solid red lines indicate the device bias relative to PSG. Solid grey lines indicate the 95% limits of agreement (±1.96 standard deviation).

Figure 2

Bland–Altman plots for device–derived and PSG–derived measures of heart rate. The y-axes represent the difference between the measures—positive values indicate devices overestimate relative to PSG, and negative values indicate devices underestimate relative to PSG. Solid red lines indicate the device bias relative to PSG. Solid grey lines indicate the 95% limits of agreement (±1.96 standard deviation). * manufacturers provided raw data; ** y-axis was adjusted in plot B to accommodate for one data point outside of the common y–axis range.

Figure 3

Bland–Altman plots for device–derived and PSG–derived measures of heart rate variability. The y–axes represent the difference between the measures—positive values indicate devices overestimate relative to PSG, and negative values indicate devices underestimate relative to PSG. Solid red lines indicate the device bias relative to PSG. Solid grey lines indicate the 95% limits of agreement (±1.96 standard deviation). * Manufacturers provided raw data.