Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Oct 27:12:821-842.
doi: 10.2147/NSS.S270705. eCollection 2020.

Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions

Jason D Stone  1 , Lauren E Rentz  1 , Jillian Forsey  1 , Jad Ramadan  1 , Rachel R Markwald  2 , Victor S Finomore  1 , Scott M Galster  1 , Ali Rezai  1 , Joshua A Hagen  1
Affiliations

Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions

Jason D Stone et al. Nat Sci Sleep. .

Abstract

Purpose: The commercial market is saturated with technologies that claim to collect proficient, free-living sleep measurements despite a severe lack of independent third-party evaluations. Therefore, the present study evaluated the accuracy of various commercial sleep technologies during in-home sleeping conditions.

Materials and methods: Data collection spanned 98 separate nights of ad libitum sleep from five healthy adults. Prior to bedtime, participants utilized nine popular sleep devices while concurrently wearing a previously validated electroencephalography (EEG)-based device. Data collected from the commercial devices were extracted for later comparison against EEG to determine degrees of accuracy. Sleep and wake summary outcomes as well as sleep staging metrics were evaluated, where available, for each device.

Results: Total sleep time (TST), total wake time (TWT), and sleep efficiency (SE) were measured with greater accuracy (lower percent errors) and limited bias by Fitbit Ionic [mean absolute percent error, bias (95% confidence interval); TST: 9.90%, 0.25 (-0.11, 0.61); TWT: 25.64%, -0.17 (-0.28, -0.06); SE: 3.49%, 0.65 (-0.82, 2.12)] and Oura smart ring [TST: 7.39%, 0.19 (0.04, 0.35); TWT: 36.29%, -0.18 (-0.31, -0.04); SE: 5.42%, 1.66 (0.17, 3.15)], whereas all other devices demonstrated a propensity to over or underestimate at least one if not all of the aforementioned sleep metrics. No commercial sleep technology appeared to accurately quantify sleep stages.

Conclusion: Generally speaking, commercial sleep technologies displayed lower error and bias values when quantifying sleep/wake states as compared to sleep staging durations. Still, these findings revealed that there is a remarkably high degree of variability in the accuracy of commercial sleep technologies, which further emphasizes that continuous evaluations of newly developed sleep technologies are vital. End-users may then be able to determine more accurately which sleep device is most suited for their desired application(s).

Keywords: consumer sleep technologies; sleep duration; sleep efficiency; sleep staging; wearables.

PubMed Disclaimer

Conflict of interest statement

Financial competing interests: none. Non-financial competing interests: none. The authors report no conflicts of interest for this work.

Figures

Figure 1
Figure 1
TST boxplots: absolute percent error by device.
Figure 2
Figure 2
(A–I) TST Bland–Altman plots for all devices.
Figure 3
Figure 3
TWT time boxplots: absolute percent error by device.
Figure 4
Figure 4
(A–G) TWT Bland–Altman plots for all devices.
Figure 5
Figure 5
SE boxplots: absolute percent error by device.
Figure 6
Figure 6
(A–G) SE Bland–Altman plots for all devices.
Figure 7
Figure 7
Light time boxplots: absolute percent error by device.
Figure 8
Figure 8
Deep time boxplots: absolute percent error by device.
Figure 9
Figure 9
REM time boxplots: absolute percent error by device.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Watson NF, Badr MS, Belenky G, et al. Recommended amount of sleep for a healthy adult: a joint consensus statement of the American academy of sleep medicine and sleep research society. Sleep. 2015. doi:10.5665/sleep.4716 - DOI - PMC - PubMed
    1. Itani O, Jike M, Watanabe N, Kaneita Y. Short sleep duration and health outcomes: a systematic review, meta-analysis, and meta-regression. Sleep Med. 2017;32:246–256. doi:10.1016/j.sleep.2016.08.006 - DOI - PubMed
    1. Kelly J, Strecker R, Bianchi M. Recent developments in home sleep monitoring devices. ISRN Neurol. 2012;2012. doi:10.5402/2012/768794 - DOI - PMC - PubMed
    1. de Zambotti M, Cellini N, Goldstone A, Colrain I, Baker F. Wearable sleep technology in clinical and research settings. Med Sci Sports Exerc. 2019;51:1538–1557. doi:10.1249/MSS.0000000000001947 - DOI - PMC - PubMed
    1. Ibañez V, Silva J, Navarro E, Cauli O. Sleep assessment devices: types, market analysis, and a critical review on accuracy and validation. Expert Rev Med Devices. 2019;16(12):1041–1052. doi:10.1080/17434440.2019.1693890 - DOI - PubMed