ReHabgame: A non-immersive virtual reality rehabilitation system with applications in neuroscience

Abstract

This paper proposes the use of a non-immersive virtual reality rehabilitation system "ReHabgame" developed using Microsoft Kinect™ and the Thalmic™ Labs Myo gesture control armband. The ReHabgame was developed based on two third-person video games that provide a feasible possibility of assessing postural control and functional reach tests. It accurately quantifies specific postural control mechanisms including timed standing balance, functional reach tests using real-time anatomical landmark orientation, joint velocity, and acceleration while end trajectories were calculated using an inverse kinematics algorithm. The game was designed to help patients with neurological impairment to be subjected to physiotherapy activity and practice postures of daily activities. The subjective experience of the ReHabgame was studied through the development of an Engagement Questionnaire (EQ) for qualitative, quantitative and Rasch model. The Monte-Carlo Tree Search (MCTS) and Random object (ROG) generator algorithms were used to adapt the physical and gameplay intensity in the ReHabgame based on the Motor Assessment Scale (MAS) and Hierarchical Scoring System (HSS). Rasch analysis was conducted to assess the psychometric characteristics of the ReHabgame and to identify if these are any misfitting items in the game. Rasch rating scale model (RSM) was used to assess the engagement of players in the ReHabgame and evaluate the effectiveness and attractiveness of the game. The results showed that the scales assessing the rehabilitation process met Rasch expectations of reliability, and unidimensionality. Infit and outfit mean squares values are in the range of (0.68-1.52) for all considered 16 items. The Root Mean Square Residual (RMSR) and the person separation reliability were acceptable. The item/person map showed that the persons and items were clustered symmetrically.

Keywords: Computer science; Rehabilitation.

Figure 1

The ReHabgame diagram and its components with MCTS algorithm.

Figure 2

Screen shots of a male and a female players with the representative avatars in the “Reach-Grasp-Release” fruits game. (a) shows the female avatar collected the peach and aiming to release it above the basket, and the male avatar collected a kiwi, (b) displays the female avatar aiming to collect the kiwi and the male avatar has collected the peach, (c) the female has collected the fruit and the male avatar is idle and (d) demonstrations the female avatar released the strawberry above the basket with green dots provided the positive feedback.

Figure 3

Screen shots of a player in the “Reach-Press-Hold” button game. (a) the avatar's right hand is engaged with the game which aiming towards the button to press its red part, (b) illustrates the avatar pressed the red button for three sec with three green dots appeared next to it which illustrates the visual feedback, (c) displays the avatar reached and pressed another button and holds it for one sec and (d) demonstrates the avatar with three seconds holding with its visual feedback.

Figure 4

Avatar in “Reach-Grasp-Release” fruits game with two steps elbow and hand position; (a) shows step 1: place the elbow on purple button in that the colour changes to green as shown in (b), it also illustrates the step 2 in that the avatar moved the hand towards the virtual lemon's location.

Figure 5

(a) is the game engine with the MCTS generator script that is attached to the generator object. (b) Illustrates the main menu of the ReHabgame. (c) shows the virtual peach generated in the player's neck high with the shoulder (sh-yaw, sh-pitch, sh-roll) and Elbow orientation values next to the player (these orientation are based on the red dots shown on the screen on avatar's right-hand side) and (d) illustrates the orange generated above player's head height.

Figure 6

(a) Illustrates the screen shot of the video footage taken from the player while standing behind the predefined line within 1.8 m from the screen and playing the fruit “Reach-Grasp-Collect” game, (b) shows the same player with the Myo armband in her arm and the video streamed from behind the player.

Figure 7

Participants engagement scores (each x represents one person), centred at a mean of zero with standard deviations of 1.0. Items are arranged in engagement order with 4 referring to “always agree” and 0 to “always disagree”.

Figure 8

The category probability curves for items test information for engagement in all categories (a) and their probabilities (b).

Figure 9

The threshold map with the distribution of person in figure (a) and item in figure (b).

Figure 10

(a) and (b) show data collected from Myo armband of the electromyography (EMG 1) generated through “Reach-Press-Hold” buttons game” from two players, (c) and (d) illustrates the elbow joint orientation collected by the Kinect from the same players.