Rapid assessment of hand reaching using virtual reality and application in cerebellar stroke

Abstract

The acquisition of sensory information about the world is a dynamic and interactive experience, yet the majority of sensory research focuses on perception without action and is conducted with participants who are passive observers with very limited control over their environment. This approach allows for highly controlled, repeatable experiments and has led to major advances in our understanding of basic sensory processing. Typical human perceptual experiences, however, are far more complex than conventional action-perception experiments and often involve bi-directional interactions between perception and action. Innovations in virtual reality (VR) technology offer an approach to close this notable disconnect between perceptual experiences and experiments. VR experiments can be conducted with a high level of empirical control while also allowing for movement and agency as well as controlled naturalistic environments. New VR technology also permits tracking of fine hand movements, allowing for seamless empirical integration of perception and action. Here, we used VR to assess how multisensory information and cognitive demands affect hand movements while reaching for virtual targets. First, we manipulated the visibility of the reaching hand to uncouple vision and proprioception in a task measuring accuracy while reaching toward a virtual target (n = 20, healthy young adults). The results, which as expected revealed multisensory facilitation, provided a rapid and a highly sensitive measure of isolated proprioceptive accuracy. In the second experiment, we presented the virtual target only briefly and showed that VR can be used as an efficient and robust measurement of spatial memory (n = 18, healthy young adults). Finally, to assess the feasibility of using VR to study perception and action in populations with physical disabilities, we showed that the results from the visual-proprioceptive task generalize to two patients with recent cerebellar stroke. Overall, we show that VR coupled with hand-tracking offers an efficient and adaptable way to study human perception and action.

Fig 1. Task and stimuli in the Visible/Invisible Hand experiment.

Each trial starts with a green cube appearing in front of the participant’s chest. After the cube is touched, the cube disappears and a pink target sphere appears along a 60-degree arc in front of the participant at arm’s length. When participant’s index finger passes through the arc, it explodes and the trial ends. A new cube appears to begin the new trial. A) In the visible hand condition, the rendering of the hand is visible during the entire trial. B) In the invisible hand condition, the rendering of the hand is invisible during the reach phase. That is, the hand rendering disappeared when the cube was touched, reappearing only at the completion of the reach movement. For a video of this experiment, see supporting information S1 Video.

Fig 2. Task and stimuli in the Memory Delay experiment.

Each trial starts with a green cube appearing in front of the participant’s chest. 500 ms after the cube is touched, the pink target sphere appears along a 60-degree arc at arm’s length. 1200 ms later, a tone indicates that a participant is free to reach out to the target. When participant’s index finger passes through the arc, it explodes and the trial ends. A new cube appears to begin the new trial. A) In the standard condition, the target remained visible for the entire trial. B) In the memory delay condition, the target disappeared 200 ms after its appearance, remaining invisible for the 1000ms before the tone was played and during the subsequent reach movement. For a video of this experiment, see supporting information S2 Video.

Fig 3. Results of the Visible/Invisible Hand experiment in healthy adults.

(A) Group-level average reaching error as a function of hand visibility in all 100 trials. Yellow: Visible-hand condition. Blue: Invisible-hand condition. Error bars denote the standard error of the mean. (B) Results for 20 individual participants as a function of hand visibility in all 100 trials. (C) Group-level average reaching error as a function of hand visibility in the first 25 trials. (D) Results for 20 individual participants as a function of hand visibility in the first 25 trials.

Fig 4. Reaching errors for each individual trial in 20 healthy adult participants in the Visible/Invisible Hand experiment.

This depiction of the data allows for visualization of data stability over the course of the experiment. Yellow: Visible-hand condition. Blue: Invisible-hand condition.

Fig 5. Results of the Memory Delay experiment in healthy adults.

(A) Group-level average reaching error as a function of memory demand in all 100 trials. Yellow: Non-delayed standard condition. Blue: Delayed condition. Error bars denote the standard error of the mean. (B) Results for 18 individual participants as a function of memory demand in all 100 trials. (C) Group-level average reaching error as a function of memory demand in the first 25 trials. (B) Results for 18 individual participants as a function of memory demand the first 25 trials.

Fig 6. Reaching errors for each individual trial in 18 healthy adult participants in the Memory Delay experiment.

This depiction of the data allows for visualization of data stability over the course of the experiment. Yellow: Non-delayed standard condition. Blue: Delayed condition.

Fig 7. Results of the Visible/Invisible Hand experiment in patients with recent cerebellar strokes.

(A) Reaching error as a function of hand visibility in all 100 trials in patient 1. Yellow: Non-delayed standard condition. Blue: Delayed condition. Error bars denote the standard error of the mean. (B) Reaching error as a function of hand visibility in the first 25 trials in patient 1. (C) Reaching error as a function of hand visibility in all 100 trials in patient 2. (D) Reaching error as a function of hand visibility in the first 25 trials in patient 2.

Fig 8. Reaching errors for each individual trial in two patients with recent cerebellar stroke.

This depiction of the data allows for visualization of data stability over the course of the experiment. Yellow: Visible-hand condition. Blue: Invisible-hand condition.