Analytical and clinical validity of wearable, multi-sensor technology for assessment of motor function in patients with Parkinson's disease in Japan

Abstract

Continuous, objective monitoring of motor signs and symptoms may help improve tracking of disease progression and treatment response in Parkinson's disease (PD). This study assessed the analytical and clinical validity of multi-sensor smartwatch measurements in hospitalized and home-based settings (96 patients with PD; mean wear time 19 h/day) using a twice-daily virtual motor examination (VME) at times representing medication OFF/ON states. Digital measurement performance was better during inpatient clinical assessments for composite V-scores than single-sensor-derived features for bradykinesia (Spearman |r|= 0.63, reliability = 0.72), tremor (|r|= 0.41, reliability = 0.65), and overall motor features (|r|= 0.70, reliability = 0.67). Composite levodopa effect sizes during hospitalization were 0.51-1.44 for clinical assessments and 0.56-1.37 for VMEs. Reliability of digital measurements during home-based VMEs was 0.62-0.80 for scores derived from weekly averages and 0.24-0.66 for daily measurements. These results show that unsupervised digital measurements of motor features with wrist-worn sensors are sensitive to medication state and are reliable in naturalistic settings.Trial Registration: Japan Pharmaceutical Information Center Clinical Trials Information (JAPIC-CTI): JapicCTI-194825; Registered June 25, 2019.

Figure 1

Study Watch and watch face showing the virtual motor examination. (a) Study Watch worn on the wrist and (b) Study Watch face showing the virtual motor examination. The virtual motor examination comprised seven structured motor tasks, which were programmed into the smartwatch and completed by patients in order from task 1 through task 7. Tasks 1 (seated rest/rest tremor), 2 (arm raise/postural tremor), 3 (hand opening/closing), 4 (arm twist/pronation/supination), and 5 (foot stomping) were completed seated. Tasks 6 (up and go) and 7 (stand still/postural stability) required the patient to stand.

Figure 2

Study design. Period 1 was optional to accommodate the most flexible schedule for patients. Patients could enroll in the study before Period 1 in an outpatient clinic or immediately before or during Period 2. At enrollment, patients were screened for eligibility, provided written informed consent, received the smartwatch, and underwent baseline study assessments. During Period 2, patients were washed out of all concomitant Parkinsonian medications. At the final outpatient clinic visit, patients underwent final assessments and returned the smartwatch. The duration of each period may have varied from the specified durations due to the convenience of having the device dispensed and returned at the study site, the timing of the patient’s scheduled visit, and the length of the hospital stay depending on their individual treatment needs; all available data were included in the analysis regardless of the duration of each period. The VME, comprising seven structured motor tasks, was conducted at scheduled times representing relatively poor symptom control (when medication was wearing off [OFF]) and good symptom control (when medication was working well [ON]). ^aDuring the 5-day inpatient period, patients could undergo an optional 2 days of assessments, which were not included in the study. ADL activities of daily living, MDS-UPDRS Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale, VME virtual motor examination.

Figure 3

Patient flow. (a) Study flow diagram and (b) individual patient participation. Patients wore the smartwatch during the 1-month pre- or post-hospitalization periods and during the 5-day inpatient assessment.

Figure 4

Analytical validity of digital measurements from single-sensor–derived features and composite V-scores. (a) Spearman rank correlation between MDS-UPDRS Part III sensor scores and neurologist-rated consensus scores on Day 3 and (b) test–retest reliability on Day 2 of the inpatient assessment.^a,b Consensus scores for the MDS-UPDRS Part III examination on Day 3 of the inpatient assessment were calculated using an in-person rating from videotaped ratings from three neurologists. The averages of all scores for the OFF and ON states on Day 3 were combined for each measure. ^aSpearman rank correlation coefficients are plotted as absolute values; original values are plotted for coefficients where the 95% CI crosses the 0 line. Correlation was considered weak for coefficients < 0.3, moderate for coefficients 0.3–0.6, and strong for coefficients > 0.6. ^bTest–retest reliability was computed from MDS-UPDRS Part III sensor scores on Day 2, in which the MDS-UPDRS examination was administered twice within a short period of time; test–retest reliability was considered poor for ICCs < 0.5, average for ICCs 0.5–0.75, good for ICCs > 0.75–0.9, and excellent for ICCs > 0.9. CI confidence interval, ICC intraclass correlation coefficient, MDS-UPDRS Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale, V-score machine-learned composite sensor scores for each motor feature.

Figure 5

Clinical validity of digital measurements from single-sensor–derived features and composite V-scores as pharmacodynamic biomarkers: levodopa effect sizes calculated using sensor data collected during the supervised MDS-UPDRS Part III and unsupervised VME on Day 3 of the inpatient assessment. Effect sizes were calculated using Cohen’s d. Effect sizes of 0.2 were considered small, 0.5 were considered medium, 0.8 large, and > 1.2 very large. The smartwatch sensor data were collected during the investigator-supervised MDS-UPDRS Part III examination and during the unsupervised VME on Day 3 of the inpatient assessment; the averages of all scores on Day 3 were combined for each measure. CI confidence interval, MDS-UPDRS Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale, V-score machine-learned composite sensor scores for each motor feature, VME virtual motor examination.

Figure 6

Test–retest reliability of VME single-sensor–derived features and composite V-scores during the post-hospitalization home-based assessment. Test–retest reliabilities are reported for daily measurements and weekly averages of all scores for the OFF and ON states during the post-hospitalization period. Test–retest reliability was considered poor for ICCs < 0.5, average for ICCs 0.5–0.75, good for ICCs > 0.75–0.9, and excellent for ICCs > 0.9. CI confidence interval, ICC intraclass correlation coefficient, V-score machine-learned composite sensor scores for each motor feature, VME virtual motor examination.

Figure 7

Development of composite V-scores from single-sensor–derived features. V-score machine-learned composite sensor scores for each motor feature.