The Use of Smartphone Keystroke Dynamics to Passively Monitor Upper Limb and Cognitive Function in Multiple Sclerosis: Longitudinal Analysis

Abstract

Background: Typing on smartphones, which has become a near daily activity, requires both upper limb and cognitive function. Analysis of keyboard interactions during regular typing, that is, keystroke dynamics, could therefore potentially be utilized for passive and continuous monitoring of function in patients with multiple sclerosis.

Objective: To determine whether passively acquired smartphone keystroke dynamics correspond to multiple sclerosis outcomes, we investigated the association between keystroke dynamics and clinical outcomes (upper limb and cognitive function). This association was investigated longitudinally in order to study within-patient changes independently of between-patient differences.

Methods: During a 1-year follow-up, arm function and information processing speed were assessed every 3 months in 102 patients with multiple sclerosis with the Nine-Hole Peg Test and Symbol Digit Modalities Test, respectively. Keystroke-dynamics data were continuously obtained from regular typing on the participants' own smartphones. Press-and-release latency of the alphanumeric keys constituted the fine motor score cluster, while latency of the punctuation and backspace keys constituted the cognition score cluster. The association over time between keystroke clusters and the corresponding clinical outcomes was assessed with linear mixed models with subjects as random intercepts. By centering around the mean and calculating deviation scores within subjects, between-subject and within-subject effects were distinguished.

Results: Mean (SD) scores for the fine motor score cluster and cognition score cluster were 0.43 (0.16) and 0.94 (0.41) seconds, respectively. The fine motor score cluster was significantly associated with the Nine-Hole Peg Test: between-subject β was 15.9 (95% CI 12.2-19.6) and within-subject β was 6.9 (95% CI 2.0-11.9). The cognition score cluster was significantly associated with the Symbol Digit Modalities Test between subjects (between-subject β -11.2, 95% CI -17.3 to -5.2) but not within subjects (within-subject β -0.4, 95% CI -5.6 to 4.9).

Conclusions: Smartphone keystroke dynamics were longitudinally associated with multiple sclerosis outcomes. Worse arm function corresponded with longer latency in typing both across and within patients. Worse processing speed corresponded with higher latency in using punctuation and backspace keys across subjects. Hence, keystroke dynamics are a potential digital biomarker for remote monitoring and predicting clinical outcomes in patients with multiple sclerosis.

Trial registration: Netherlands Trial Register NTR7268; https://trialsearch.who.int/Trial2.aspx?TrialID=NTR7268.

Keywords: cognition; digital technology; keystroke dynamics; mobile app; multiple sclerosis; outpatient monitoring; smartphone; typing; upper extremity.

Figure 1

Overview of the Neurokeys keyboard (A) and a schematic representation of the keystroke dynamics features and clusters (B). APP: after-punctuation pause; CD: correction duration; FT: flight time; HT: hold time; post-CS: postcorrection slowing; PPL: press-press latency; pre-CS: precorrection slowing; RRL: release-release latency.

Figure 2

Bar graph depicting the retention rate (left y-axis, “user percentage”) of patients per month with superimposed box plots of the number of daily keystroke events (right y-axis, “event count”). The values above the bars show the retention rates as percentages.

Figure 3

Scatter plots and linear mixed model fit for the Nine-Hole Peg Test and fine motor score cluster by covariates (sex and age), with random intercepts on subject level, and the number of days that constituted the keystroke cluster data points. FMSC: fine motor score cluster; NHPT: Nine-Hole Peg Test.

Figure 4

Scatter plots and linear mixed model fit for Symbol Digit Modalities Test and cognition score cluster by covariates (level of education, age, and sex) and random intercepts on subject level. SDMT: symbol digit modalities test; CSC: cognition score cluster.