Efficient embedded sleep wake classification for open-source actigraphy

Abstract

This study presents a thorough analysis of sleep/wake detection algorithms for efficient on-device sleep tracking using wearable accelerometric devices. It develops a novel end-to-end algorithm using convolutional neural network applied to raw accelerometric signals recorded by an open-source wrist-worn actigraph. The aim of the study is to develop an automatic classifier that: (1) is highly generalizable to heterogenous subjects, (2) would not require manual features' extraction, (3) is computationally lightweight, embeddable on a sleep tracking device, and (4) is suitable for a wide assortment of actigraphs. Hereby, authors analyze sleep parameters, such as total sleep time, waking after sleep onset and sleep efficiency, by comparing the outcomes of the proposed algorithm to the gold standard polysomnographic concurrent recordings. The relatively substantial agreement (Cohen's kappa coefficient, median, equal to 0.78 ± 0.07) and the low-computational cost (2727 floating-point operations) make this solution suitable for an on-board sleep-detection approach.

Figure 1

Simplified CNN architecture representation, named lightCNNA. For each layer, the layer type used and its main hyperparameters are reported.

Figure 2

A comparison of the main sleep metrics calculated using PSG, lightCNNA and other alternative machine learning models for each subject included in the Leave One Subject Out (LOSO) validation procedure.

Figure 3

Estimation error for each sleep metric. Bland–Altman plots showing the difference in sleep metrics obtained using each of the machine learning models and the lightCNNA (highlighted by a black box) with respect to the PSG reference. The first row shows total sleep time for each model, the second waking after sleep onset, and the third sleep efficiency. Solid red lines identify the a priori acceptable limits of agreement of ± 30 min difference on TST. A dashed black line shows the zero error or perfect agreement with the PSG. Scaling is kept constant for each figure. For each axis we show the minimum and maximum values. The values reported on the y-axis are computed as PSG reference value minus the value computed by the alternative method.

Figure 4

Comparison of kappa and F1 performance metrics scored by the lightCNNA model and by all other machine learning approaches. All data were computed using the LOSO approach Each point represents a subject. The number of subjects with a F1 score below 0.8 or above 0.9 is shown in panel c, at the left and right side of each boxplot. The corresponding number of subjects scoring kappa above 0.8 or below 0.6 is shown in panel d.

Figure 5

Optimization of binarization threshold of lightCNNA output. (a) shows the effect on models performances of the variation of the binarization threshold. (b) highlights the point in which the classifier maximizes concordance, by showing the minimum absolute difference between specificity and sensitivity (c). (d) shows the value of the best threshold estimated for each subject during the LOSO validation.

Figure 6

Variation of binarization threshold on single LOSO subject data. (a) Effect of variation of the binarization threshold level on CKC for each LOSO subject. Performance achieved using a specific binarization treshold: in (b) CKC and concordance, in (c) specificity and sensitivity; solid lines represent median values, the shaded area shows the corresponding mean amplitude deviance.