Sleep Validity of a Non-Contact Bedside Movement and Respiration-Sensing Device

Abstract

Study objectives: To assess the sleep detection and staging validity of a non-contact, commercially available bedside bio-motion sensing device (S+, ResMed) and evaluate the impact of algorithm updates.

Methods: Polysomnography data from 27 healthy adult participants was compared epoch-by-epoch to synchronized data that were recorded and staged by actigraphy and S+. An update to the S+ algorithm (common in the rapidly evolving commercial sleep tracker industry) permitted comparison of the original (S+V1) and updated (S+V2) versions.

Results: Sleep detection accuracy by S+V1 (93.3%), S+V2 (93.8%), and actigraphy (96.0%) was high; wake detection accuracy by each (69.6%, 73.1%, and 47.9%, respectively) was low. Higher overall S+ specificity, compared to actigraphy, was driven by higher accuracy in detecting wake before sleep onset (WBSO), which differed between S+V2 (90.4%) and actigraphy (46.5%). Stage detection accuracy by the S+ did not exceed 67.6% (for stage N2 sleep, by S+V2) for any stage. Performance is compared to previously established variance in polysomnography scored by humans: a performance standard which commercial devices should ideally strive to reach.

Conclusions: Similar limitations in detecting wake after sleep onset (WASO) were found for the S+ as have been previously reported for actigraphy and other commercial sleep tracking devices. S+ WBSO detection was higher than actigraphy, and S+V2 algorithm further improved WASO accuracy. Researchers and clinicians should remain aware of the potential for algorithm updates to impact validity.

Commentary: A commentary on this article appears in this issue on page 935.

Keywords: actigraphy; consumer device; sleep; validation.

Figure 1. Total sleep time disagreement between measurements.

Bland-Altman concordance between total sleep time as measured by PSG and the S+ device (A) or actigraphy (B). Solid lines indicate average discrepancy, dashed and dotted lines indicate ± 1 standard deviation, and symbols correspond to the number of minutes differing for each participant. On the ordinate, a positive difference indicates overestimation and a negative difference underestimation relative to PSG. The range of participant sleep time in minutes, according to RPSGT-scored PSG, is on the abscissa. Act = actigraphy, Avg = average, Diff = difference, PSG = polysomnography, S+V1 = algorithm version 1, S+V2 = algorithm version 2, SD = standard deviation.

Figure 2. Total wake time disagreement between measurements.

Bland-Altman concordance between total wake time as measured by PSG and the S+ device (A) or actigraphy (B). Solid lines indicate average discrepancy, dashed and dotted lines indicate ± 1 standard deviation, and symbols correspond to the number of minutes differing for each participant. On the ordinate, a positive difference indicates overestimation and a negative difference underestimation relative to PSG. The range of participant wake time in minutes, according to RPSGT-scored PSG, is on the abscissa. Act = actigraphy, Avg = average, Diff = difference, PSG = polysomnography, S+V1 = algorithm version 1, S+V2 = algorithm version 2, SD = standard deviation.

Figure 3. Sleep staging disagreement between measurements.

Bland-Altman concordance between the S+ device and PSG when recognizing both stage N1 and N2 sleep as light sleep (A), stage N3 sleep as deep sleep (B), and REM sleep (C). Solid lines indicate average discrepancy, dashed and dotted lines indicate ± 1 standard deviation, and symbols correspond to the number of minutes differing for each participant. On the ordinate, a positive difference indicates overestimation and a negative difference underestimation relative to PSG. The range of participant light, deep, or REM sleep time in minutes, according to RPSGT-scored PSG, is on the abscissa. Avg = average, Diff = difference, PSG = polysomnography, S+V1 = algorithm version 1, S+V2 = algorithm version 2, SD = standard deviation.

Figure 4. Epoch-by-epoch agreement with PSG.

Percent agreement either between a sleep monitoring device (S+V1, S+V2, or actigraphy) and a corresponding PSG record (scored by an RPSGT) in this study, or between PSG records scored by multiple registered technologists (indicated by dashed gray lines; inter-scorer reliability). Error bars indicate standard deviation (n = 22 in all device groups). Overall sensitivity and specificity results reflect post hoc pairwise outcomes after significant omnibus repeated-measures ANOVA. Stage-specific sensitivity results reflect outcomes of within-subjects t tests. * P < .05, ** P < .01. PSG = polysomnography, REM = rapid eye movement sleep, S+V1 = algorithm version 1, S+V2 = algorithm version 2.