Feedback from HTC Vive Sensors Results in Transient Performance Enhancements on a Juggling Task in Virtual Reality

Abstract

Virtual reality headsets, such as the HTC Vive, can be used to model objects, forces, and interactions between objects with high perceived realism and accuracy. Moreover, they can accurately track movements of the head and the hands. This combination makes it possible to provide subjects with precise quantitative feedback on their performance while they are learning a motor task. Juggling is a challenging motor task that requires precise coordination of both hands. Professional jugglers throw objects so that the arc peaks just above head height, and they time their throws so that the second ball is thrown when the first ball reaches its peak. Here, we examined whether it is possible to learn to juggle in virtual reality and whether the height and the timing of the throws can be improved by providing immediate feedback derived from the motion sensors. Almost all participants became better at juggling in the ~30 min session: the height and timing of their throws improved and they dropped fewer balls. Feedback on height, but not timing, improved performance, albeit only temporarily.

Keywords: HTC Vive; feedback; juggling; learning; timing; virtual reality.

Figure 1

Hardware and user interface. The HTC Vive headset and the base station is shown in (A); the HTC hand controller is shown in (B). In (C), the interface of Unity, which was used to create the juggling simulation, is shown. (D) shows the TV within the VR environment that subjects received instructions on. (E) shows the hand and the ball used in the juggling simulator. (F) shows a picture of the two-ball juggling tutorial that subjects were shown in VR and asked to imitate. The height and timing feedback given to subjects in the feedback group is shown in (G,H), respectively.

Figure 2

Training protocol. All participants first completed an initial practice phase in which participants got accustomed to the VR environment and learned to throw and catch one ball. Next, all participants completed 50 throws with two balls. After these 50 throws, the feedback group received height feedback on the next 50 trials and then timing feedback on the next 50 trials. The control group simply continued to practice without any quantitative input. The session finished with another 50 throws in which no feedback was given. See Figure 1 for larger versions of the screenshots.

Figure 3

Drops. The figure shows the number of drops before each successful two-ball throw for all participants. Each dot represents the number of drops before one successful attempt for one participant. The line in the middle shows the median value. There is a clear decline in the number drops before each successful attempt as the participants spend more time in the simulator, suggesting that participants improved their performance over time. The boxplot to the right shows the distribution of change in drops. For each participant, the average number of drops on the last 10 trials (191–200) was subtracted from the number of drops on throws 41–50.

Figure 4

Height and timing errors (mean ± SEM) for participants receiving feedback (top row, colored lines/dots) and controls (bottom row, black lines). Panels A–E shows the difference between the optimal height of throws and the height of the actual throws with the right hand (A,B), and the left hand (C,D). Panels (E,F) shows the timing error. The color represents the stage of the experiment: blue represent baseline trials, before the participant received any feedback; red represent throws on trials where the participants received feedback on the height of their throws; yellow represent trials where participants received timing feedback; and purple represent post-feedback trials where participants did not receive any feedback.

Figure 5

Effect of height and timing feedback. (A–C) displays the height/timing of the throws before feedback, at the end of the feedback sessions, and on the post-test. (D–F) displays how much the height/timing of the throws improved during the feedback session. This was calculated by taking the average error on the last 10 trials before the feedback session (throws 41–50) and subtracting the average error on the last 10 trials of the feedback session (41–50 for height; 91–100 for timing), and then comparing the change to the change in the control group. Participants receiving feedback on height improved the height of their throws more than those who did not receive feedback. However, this was not the case for the timing feedback.