Angle Assessment for Upper Limb Rehabilitation: A Novel Light Detection and Ranging (LiDAR)-Based Approach

Abstract

The accurate measurement of joint angles during patient rehabilitation is crucial for informed decision making by physiotherapists. Presently, visual inspection stands as one of the prevalent methods for angle assessment. Although it could appear the most straightforward way to assess the angles, it presents a problem related to the high susceptibility to error in the angle estimation. In light of this, this study investigates the possibility of using a new approach to angle calculation: a hybrid approach leveraging both a camera and LiDAR technology, merging image data with point cloud information. This method employs AI-driven techniques to identify the individual and their joints, utilizing the cloud-point data for angle computation. The tests, considering different exercises with different perspectives and distances, showed a slight improvement compared to using YOLO v7 for angle calculation. However, the improvement comes with higher system costs when compared with other image-based approaches due to the necessity of equipment such as LiDAR and a loss of fluidity during the exercise performance. Therefore, the cost-benefit of the proposed approach could be questionable. Nonetheless, the results hint at a promising field for further exploration and the potential viability of using the proposed methodology.

Keywords: Artificial Intelligence; LiDAR; join angle measurement; motion capture; robotic rehabilitation.

Figure 1

Proposed system, representing the different coordinate frame.

Figure 2

Example of the position of the sensors and the patient during the data collection. (A) LiDAR field of view. (B) Camera field of view.

Figure 3

Flowchart of the proposed approach.

Figure 4

Example of the LiDAR measurement process merge: (A) the initial measurement, (B) the fusion between the initial measurement and the second, (C) fusion with the third measurement, (D) fusion with the fourth, and (E) the final fusion of the points cloud, with each color representing one measurement.

Figure 5

Example of person identification using YOLO v7.

Figure 6

Example of the point cloud fitting in the person on the image.

Figure 7

Example of the joints identified in the person. Green points related to the right wrist. Red points related to the right shoulder. Blue points related to the right hip.

Figure 8

Example of joint identification, with the equivalent fitted point cloud for the elbow flexion ($R_7$) at a 4−m distance and an oblique view: (.1) at a 130° angle; (.2) at a 96° angle; (.3) at a 70° angle. (A) The points marked in green, red, and blue correspond to the shoulder, elbow, and wrist, respectively. (B) colors represents distinct data captured by the LiDAR at different iteration.

Figure 9

Demonstration of the three performed exercises in frontal view, with the targeted rotation joint. (A) Shoulder abduction ($R_A$), (B) shoulder flexion ($R_F$), (C) elbow flexion ($R_7$), and (•) targeted rotation joint.

Figure 10

Demonstration of the three performed exercises in lateral view. (A) Shoulder abduction (R_A), (B) shoulder flexion (R_F), and (C) elbow flexion (R₇).

Figure 11

Wearable module attached to the UR3 with its reference coordinate frame and the coordinate frame selected for $\theta$ calculation.

Figure 12

UR3 joint movements compared to the angle measured using the IMU.

Figure 13

Shoulder abduction ($R_A$) results with three approaches compared to a ground truth reference at 4 and 3 m distances. (Green) Frontal View. (Blue) Oblique view. (Orange) Lateral view.

Figure 14

MAE, mean, and standard deviation for the $R_A$ rotation angle with different views and distances. (Green) Frontal View. (Blue) Oblique view. (Orange) Lateral view.

Figure 15

Shoulder flexion ($R_F$) results with three approaches compared to a ground truth reference at 4 and 3 m distances. (Green) Frontal view. (Blue) Oblique view. (Orange) Lateral view.

Figure 16

MAE, mean, and standard deviation for the $R_F$ rotation angle with different views and distances. (Green) Frontal view. (Blue) Oblique view. (Orange) Lateral view.

Figure 17

Elbow Flexion ($R_7$) results with three approaches compared to a ground truth reference at 4 and 3 m distances. (Green) Frontal view. (Blue) Oblique view. (Orange) Lateral view.

Figure 18

MAE, Mean, and Standard deviation for the $R_7$ rotation angle with different views and distances. (Green) Frontal view. (Blue) Oblique view. (Orange) Lateral view.

Figure 19

Histogram, probability plot, mean, MAE, and standard deviation of errors for the different approaches, computed with the data from all the exercises, views, and distances (excluding lateral view).