HemoKinect: A Microsoft Kinect V2 Based Exergaming Software to Supervise Physical Exercise of Patients with Hemophilia

Abstract

Patients with hemophilia need to strictly follow exercise routines to minimize their risk of suffering bleeding in joints, known as hemarthrosis. This paper introduces and validates a new exergaming software tool called HemoKinect that intends to keep track of exercises using Microsoft Kinect V2's body tracking capabilities. The software has been developed in C++ and MATLAB. The Kinect SDK V2.0 libraries have been used to obtain 3D joint positions from the Kinect color and depth sensors. Performing angle calculations and center-of-mass (COM) estimations using these joint positions, HemoKinect can evaluate the following exercises: elbow flexion/extension, knee flexion/extension (squat), step climb (ankle exercise) and multi-directional balance based on COM. The software generates reports and progress graphs and is able to directly send the results to the physician via email. Exercises have been validated with 10 controls and eight patients. HemoKinect successfully registered elbow and knee exercises, while displaying real-time joint angle measurements. Additionally, steps were successfully counted in up to 78% of the cases. Regarding balance, differences were found in the scores according to the difficulty level and direction. HemoKinect supposes a significant leap forward in terms of exergaming applicability to rehabilitation of patients with hemophilia, allowing remote supervision.

Keywords: Kinect; exergaming; hemophilia; physical exercise; rehabilitation.

Figure 1

HemoKinect flowchart describing the implemented methodology. The depth frames are processed to obtain the skeletal data from them using a machine learning algorithm, in particular a random decision forest [38]. The skeletal data are then mapped to the color image space using methods coded in Microsoft’s SDK. Then, the overlay image is obtained by drawing the bodies on top of the color image and adding the calculated angles’ achievement indicators. The process continues until the execution is halted. Finally, summary reports and graphs of the exercise are generated.

Figure 2

Screenshots of HemoKinect during the different exercises: (a) simultaneous elbow flexion exercise, without load; (b) knee flexion exercise (squat); (c) ankle exercise (step climb); (d) elbow flexion exercise in two-player mode. The repetition count for each exercise and individual left/right joint of interest is displayed in the respective left/right corner of the screen. In two-player mode, the repetition count for the second player appears stacked directly below the first players’ count, in a different color. In this last screenshot, the real-time angle measurement of the joints of interest is displayed, as well, in a smaller font, below the respective score box.

Figure 3

Sample screenshots of HemoKinect’s balance exercise. The player’s COM is represented by a blue circumference. The current target position is represented by a green circle of radius 1.5 cm, whose edge changes color from red to green when the player’s COM is inside it. The figure shows: (a) the player has not reached the current target balance position, and the circle’s edge color is red; (b) the player reaches the current target balance position (NE), and the edge color changes to green; (c) the player reaches the starting position (idlePos) between two successive balance positions.

Figure 4

Graphs representing the joint angle (°) fluctuation of different exercises over time (s): simultaneous elbow flexions and extensions (a) without load and (b) with load; alternate elbow flexions and extensions (c) without load and (d) with load; (e) knee flexions and extensions (squats) and (f) right ankle angle fluctuation when performing four right-foot step climbs and descents.

Figure 5

Average balance results (%) on the hemophilic population for all eight cardinal and intermediate positions: (a) box plot of average balance performance per patient including all three levels; (b) box plot of average balance performance per level for all patients; and (c) box plot of the average balance performance per cardinal direction for all patients and levels.