Influence of virtual reality and task complexity on digital health metrics assessing upper limb function

Abstract

Background: Technology-based assessments using 2D virtual reality (VR) environments and goal-directed instrumented tasks can deliver digital health metrics describing upper limb sensorimotor function that are expected to provide sensitive endpoints for clinical studies. Open questions remain about the influence of the VR environment and task complexity on such metrics and their clinimetric properties.

Methods: We aim to investigate the influence of VR and task complexity on the clinimetric properties of digital health metrics describing upper limb function. We relied on the Virtual Peg Insertion Test (VPIT), a haptic VR-based assessment with a virtual manipulation task. To evaluate the influence of VR and task complexity, we designed two novel tasks derived from the VPIT, the VPIT-2H (VR environment with reduced task complexity) and the PPIT (physical task with reduced task complexity). These were administered in an observational longitudinal study with 27 able-bodied participants and 31 participants with multiple sclerosis (pwMS, VPIT and PPIT only) and the value of kinematic and kinetic metrics, their clinimetric properties, and the usability of the assessment tasks were compared.

Results: Intra-participant variability strongly increased with increasing task complexity (coefficient of variation + 56%) and was higher in the VR compared to the physical environment (+ 27%). Surprisingly, this did not translate into significant differences in the metrics' measurement error and test-retest reliability across task conditions (p > 0.05). Responsiveness to longitudinal changes in pwMS was even significantly higher (effect size + 0.35, p < 0.05) for the VR task with high task complexity compared to the physical instrumented task with low task complexity. Increased inter-participant variability might have compensated for the increased intra-participant variability to maintain good clinimetric properties. No significant influence of task condition on concurrent validity was present in pwMS. Lastly, pwMS rated the PPIT with higher usability than the VPIT (System Usability Scale + 7.5, p < 0.05).

Conclusion: The metrics of both the VR haptic- and physical task-based instrumented assessments showed adequate clinimetric properties. The VR haptic-based assessment may be superior when longitudinally assessing pwMS due to its increased responsiveness. The physical instrumented task may be advantageous for regular clinical use due to its higher usability. These findings highlight that both assessments should be further validated for their ideal use-cases.

Fig. 1

Assessment platforms VPIT (A), PPIT (B), and virtual display of the VPIT-2H (C). These are used to study the influence of task complexity and virtual reality on the clinimetric properties of digital health metrics

Fig. 2

Overview of study protocol (A, B) and analysis approach (C). Able-bodied participants were tested on the VPIT, PPIT, and VPIT-2H in a test–retest protocol. Participants with Multiple Sclerosis were tested with the VPIT and PPIT at admission and discharge within a 3-week neurorehabilitation program. The analysis focused on a comparison between the three experimental conditions (VPIT, PPIT, VPIT-2H) for the clinimetric properties (primary aim) and to gain a behavioral understanding of the effect of experimental conditions and describe the usability of the assessments (secondary aim). EDSS: Expanded Disability Status Scale. NHPT: Nine Hole Peg Test. BBT: Box and Block Test. CoV: Coefficient of Variation. ICC: Intra-class correlation coefficient. SRD: Smallest Real Difference. SRM: Standardized response mean. # > SRD: Number of individuals with changes larger than the SRD. SUS: System Usability Scale

Fig. 3

Able-bodied participants. Visualization of an example metric across all conditions (A, grey lines connect individual participants), its coefficient of variation (CoV, B, grey lines connect individual participants), and the CoV for all metrics and conditions (C). In addition, the test–retest reliability of metrics across all conditions (D, grey lines connect individual metrics) and the usability outcomes (E, grey lines connect individual participants) are depicted. The middle, long horizontal bar represents the median and the shorter horizontal bars or end of the filled box the 25th- and 75th-percentile. The whiskers in C represent the minimum and maximum value within 1.5 times the interquartile range. A.u. arbitrary units. *p < 0.05. **p < 0.01

Fig. 4

Participants with multiple sclerosis. Visualization of an example metric for all participants with multiple sclerosis across all conditions (A), its coefficient of variation (CoV, B), and the CoV for all metrics and conditions (C). In addition, the responsiveness of all metrics across all conditions (D) and the usability outcomes (E) are depicted. Detailed legend in Fig. 3