Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Nov;54(11):2907-2926.
doi: 10.1007/s40279-024-02077-2. Epub 2024 Jul 30.

Keeping Pace with Wearables: A Living Umbrella Review of Systematic Reviews Evaluating the Accuracy of Consumer Wearable Technologies in Health Measurement

Cailbhe Doherty  1   2 , Maximus Baldwin  3   4   5 , Alison Keogh  4   6 , Brian Caulfield  3   4 , Rob Argent  4   7
Affiliations

Keeping Pace with Wearables: A Living Umbrella Review of Systematic Reviews Evaluating the Accuracy of Consumer Wearable Technologies in Health Measurement

Cailbhe Doherty et al. Sports Med. 2024 Nov.

Abstract

Background: Consumer wearable technologies have become ubiquitous, with clinical and non-clinical populations leveraging a variety of devices to quantify various aspects of health and wellness. However, the accuracy with which these devices measure biometric outcomes such as heart rate, sleep and physical activity remains unclear.

Objective: To conduct a 'living' (i.e. ongoing) evaluation of the accuracy of consumer wearable technologies in measuring various physiological outcomes.

Methods: A systematic search of the literature was conducted in the following scientific databases: MEDLINE via PubMed, Embase, Cinahl and SPORTDiscus via EBSCO. The inclusion criteria required systematic reviews or meta-analyses that evaluated the validation of consumer wearable devices against accepted reference standards. In addition to publication details, review protocol, device specifics and a summary of the authors' results, we extracted data on mean absolute percentage error (MAPE), pooled absolute bias, intraclass correlation coefficients (ICCs) and mean absolute differences.

Results: Of 904 identified studies through the initial search, 24 systematic reviews met our inclusion criteria; these systematic reviews included 249 non-duplicate validation studies of consumer wearable devices involving 430,465 participants (43% female). Of the commercially available wearable devices released to date, approximately 11% have been validated for at least one biometric outcome. However, because a typical device can measure a multitude of biometric outcomes, the number of validation studies conducted represents just 3.5% of the total needed for a comprehensive evaluation of these devices. For heart rate, wearables showed a mean bias of ± 3%. In arrhythmia detection, wearables exhibited a pooled sensitivity and specificity of 100% and 95%, respectively. For aerobic capacity, wearables significantly overestimated VO2max by ± 15.24% during resting tests and ± 9.83% during exercise tests. Physical activity intensity measurements had a mean absolute error ranging from 29 to 80%, depending on the intensity of the activity being undertaken. Wearables mostly underestimated step counts (mean absolute percentage errors ranging from - 9 to 12%) and energy expenditure (mean bias = - 3 kcal per minute, or - 3%, with error ranging from - 21.27 to 14.76%). For blood oxygen saturation, wearables showed a mean absolute difference of up to 2.0%. Sleep measurement showed a tendency to overestimate total sleep time (mean absolute percentage error typically > 10%).

Conclusions: While consumer wearables show promise in health monitoring, a conclusive assessment of their accuracy is impeded by pervasive heterogeneity in research outcomes and methodologies. There is a need for standardised validation protocols and collaborative industry partnerships to enhance the reliability and practical applicability of wearable technology assessments.

Prospero id: CRD42023402703.

PubMed Disclaimer

Conflict of interest statement

Declarations Author Contributions Doherty C. and Argent R. designed and supervised the study. Doherty C. and Baldwin M. screened the literature and extracted the data. Doherty C. and Baldwin M. conducted the analyses, had full access to data and drafted the manuscript. All authors approved the project design, and critically reviewed and contributed to the final version of this work and approved it. Transparency Statement The authors declare the manuscript is an honest, accurate and transparent account of the study being reported; no important aspects of the study have been omitted; and any discrepancies from the study as originally planned have been explained in published protocol and supplementary methods. Funding This project was funded by Science Foundation Ireland’s National Challenge Fund (Grant ID: 22/NCF/FD/10949). Conflict of Interest The authors have no conflicts of interest with the content of this article. Data Availability Data for this living systematic review will be made available on Open Science Framework (OSF; https://osf.io/fqvms/?view_only=e49e7c42dfd3475db5cf2da3b15e4b3f).

Figures

Fig. 1
Fig. 1
Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram
Fig. 2
Fig. 2
Biometric outcomes and wearable devices evaluated in each review
See this image and copyright information in PMC

References

    1. Lupton D. The quantified self: a sociology of self-tracking. Sociol Health Illn. 2016;39:1557–71. - DOI
    1. Piwek L, Ellis DA, Andrews S, Joinson A. The rise of consumer health wearables: promises and barriers. PLoS Med. 2016;13(2): e1001953. - DOI - PMC - PubMed
    1. GVR. Wearable technology market size, share & trends analysis report by product (head & eyewear, wristwear), by application (consumer electronics, healthcare), by region (Asia Pacific, Europe), And Segment Forecasts, 2023–2030: Grand View Research; 2023. Report No.: 978-1-68038-165-8.
    1. ACSM. Wearable technology named top fitness trend for 2024. 2024 [cited; https://www.acsm.org/education-resources/trending-topics-resources/acsm-....
    1. Carpenter A, Frontera A. Smart-watches: a potential challenger to the implantable loop recorder? Europace. 2016;18(6):791–3. - DOI - PubMed

Publication types