Sensor Validation and Diagnostic Potential of Smartwatches in Movement Disorders

Abstract

Smartwatches provide technology-based assessments in Parkinson's Disease (PD). It is necessary to evaluate their reliability and accuracy in order to include those devices in an assessment. We present unique results for sensor validation and disease classification via machine learning (ML). A comparison setup was designed with two different series of Apple smartwatches, one Nanometrics seismometer and a high-precision shaker to measure tremor-like amplitudes and frequencies. Clinical smartwatch measurements were acquired from a prospective study including 450 participants with PD, differential diagnoses (DD) and healthy participants. All participants wore two smartwatches throughout a 15-min examination. Symptoms and medical history were captured on the paired smartphone. The amplitude error of both smartwatches reaches up to 0.005 g, and for the measured frequencies, up to 0.01 Hz. A broad range of different ML classifiers were cross-validated. The most advanced task of distinguishing PD vs. DD was evaluated with 74.1% balanced accuracy, 86.5% precision and 90.5% recall by Multilayer Perceptrons. Deep-learning architectures significantly underperformed in all classification tasks. Smartwatches are capable of capturing subtle tremor signs with low noise. Amplitude and frequency differences between smartwatches and the seismometer were under the level of clinical significance. This study provided the largest PD sample size of two-hand smartwatch measurements and our preliminary ML-evaluation shows that such a system provides powerful means for diagnosis classification and new digital biomarkers, but it remains challenging for distinguishing similar disorders.

Keywords: Parkinson’s disease; artificial intelligence; movement disorders; smartwatches.

Figure 1

Processing steps include smartwatch validation with seismometers and patient data generation via an observational study for diagnostic machine learning.

Figure 2

Experimental setup of the sensor validation experiment. Apple Watches Series 3 and 4 and a Nanometrics Trillium Compact seismometer were placed on a vertical vibration table. The table simultaneously accelerated the devices by oscillatory motions with tremor-typical frequencies and amplitudes. Both watches were connected to Apple iPhones (not in this figure) via Bluetooth, where the measurement data were stored. The seismometer data were collected on a digitizer (not in this figure) that the device was connected to.

Figure 3

Overview of data analytics pipeline. SVM = support vector machine with radial basis function. CatB = CatBoost, MLP = multi-layer perceptron with two hidden layers, DL = deep-learning architecture.

Figure 4

Differences between the dominant frequency measured by the Trillium Compact seismometer and Apple Smartwatches Series 3 and 4 in a shaker table experiment. The experiment was conducted on two different days with the Apple watch Series 3. The figure shows the difference in dominant frequency (a) using the pre-defined watches’ sample rate and (b) using the watches’ actual sample rate (calculated with watch-specific timestamps) for spectral calculations. Data points that have exactly the same value lie on top of each other in the plot. To show the effect of amplitude on these frequency differences, some measurements were repeated by keeping the shaking table frequency constant and varying the shaking table amplitude.

Figure 5

(a) Self noise of watches and seismometer and (b) power spectral density (PSD) of watches, captured during a 20-s period without vibration of the shaker table. The power spectral density shows that the noise of the smartwatches had a similar intensity at all frequencies covered. However, Apple Watch 4 had a slightly higher self noise.

Figure 6

Measured oscillation amplitude of the seismometer and the watches are plotted against each other. The standard deviations of the amplitude mean values are plotted as error bars (horizontal error bar: seismometer values, vertical error bar: watch values). The grey line corresponds to a perfect agreement between the oscillation amplitude measured by the watches and the seismometer.

Figure 7

Importance of the features based on statistical information gain by CatBoost.