Accuracy of 12 Wearable Devices for Estimating Physical Activity Energy Expenditure Using a Metabolic Chamber and the Doubly Labeled Water Method: Validation Study

Abstract

Background: Self-monitoring using certain types of pedometers and accelerometers has been reported to be effective for promoting and maintaining physical activity (PA). However, the validity of estimating the level of PA or PA energy expenditure (PAEE) for general consumers using wearable devices has not been sufficiently established.

Objective: We examined the validity of 12 wearable devices for determining PAEE during 1 standardized day in a metabolic chamber and 15 free-living days using the doubly labeled water (DLW) method.

Methods: A total of 19 healthy adults aged 21 to 50 years (9 men and 10 women) participated in this study. They followed a standardized PA protocol in a metabolic chamber for an entire day while simultaneously wearing 12 wearable devices: 5 devices on the waist, 5 on the wrist, and 2 placed in the pocket. In addition, they spent their daily lives wearing 12 wearable devices under free-living conditions while being subjected to the DLW method for 15 days. The PAEE criterion was calculated by subtracting the basal metabolic rate measured by the metabolic chamber and 0.1×total energy expenditure (TEE) from TEE. The TEE was obtained by the metabolic chamber and DLW methods. The PAEE values of wearable devices were also extracted or calculated from each mobile phone app or website. The Dunnett test and Pearson and Spearman correlation coefficients were used to examine the variables estimated by wearable devices.

Results: On the standardized day, the PAEE estimated using the metabolic chamber (PAEEcha) was 528.8±149.4 kcal/day. The PAEEs of all devices except the TANITA AM-160 (513.8±135.0 kcal/day; P>.05), SUZUKEN Lifecorder EX (519.3±89.3 kcal/day; P>.05), and Panasonic Actimarker (545.9±141.7 kcal/day; P>.05) were significantly different from the PAEEcha. None of the devices was correlated with PAEEcha according to both Pearson (r=-.13 to .37) and Spearman (ρ=-.25 to .46) correlation tests. During the 15 free-living days, the PAEE estimated by DLW (PAEEdlw) was 728.0±162.7 kcal/day. PAEE values of all devices except the Omron Active style Pro (716.2±159.0 kcal/day; P>.05) and Omron CaloriScan (707.5±172.7 kcal/day; P>.05) were significantly underestimated. Only 2 devices, the Omron Active style Pro (r=.46; P=.045) and Panasonic Actimarker (r=.48; P=.04), had significant positive correlations with PAEEdlw according to Pearson tests. In addition, 3 devices, the TANITA AM-160 (ρ=.50; P=.03), Omron CaloriScan (ρ=.48; P=.04), and Omron Active style Pro (ρ=.48; P=.04), could be ranked in PAEEdlw.

Conclusions: Most wearable devices do not provide comparable PAEE estimates when using gold standard methods during 1 standardized day or 15 free-living days. Continuous development and evaluations of these wearable devices are needed for better estimations of PAEE.

Keywords: accelerometry; doubly labeled water; energy expenditure; indirect calorimetry; physical activity.

Figure 1

Differences between PAEEcha (physical activity energy expenditure) or PAEEdlw and each PAEEdev on a standardized day and 15 free-living days. Note: (a) The correlations between PAEEcha or PAEEdlw in x axis and delta as PAEEdev - PAEEcha or PAEEdev - PAEEdlw in y axis were showed. (b) P<.05 vs PAEEcha or PAEEdlw. (c) P<.05 in correlations. PAEE: physical activity energy expenditure; MAPE: mean absolute percentage error; avg: average; SD: standard deviation.

Figure 2

Correlation between PAEEcha (physical activity energy expenditure) and PAEEdev during 1 standardized day. Scattered plots between PAEEcha (x-axis) and PAEEdev (y-axis) during 1 standardized day. There was no significant correlation according to Pearson and Spearman tests. n.s.: nonsignificant.

Figure 3

Correlation between PAEEdlw (physical activity energy expenditure) and PAEEdev during 15 free-living days. Scatter plots for PAEEdlw (x-axis) and PAEEdev (y-axis) during 15 free-living days. Upper and lower values for r and ρ resulting from Pearson and Spearman correlation tests, respectively, are shown. n.s.: nonsignificant.