Review of Vision-Based Environmental Perception for Lower-Limb Exoskeleton Robots

Abstract

The exoskeleton robot is a wearable electromechanical device inspired by animal exoskeletons. It combines technologies such as sensing, control, information, and mobile computing, enhancing human physical abilities and assisting in rehabilitation training. In recent years, with the development of visual sensors and deep learning, the environmental perception of exoskeletons has drawn widespread attention in the industry. Environmental perception can provide exoskeletons with a certain level of autonomous perception and decision-making ability, enhance their stability and safety in complex environments, and improve the human-machine-environment interaction loop. This paper provides a review of environmental perception and its related technologies of lower-limb exoskeleton robots. First, we briefly introduce the visual sensors and control system. Second, we analyze and summarize the key technologies of environmental perception, including related datasets, detection of critical terrains, and environment-oriented adaptive gait planning. Finally, we analyze the current factors limiting the development of exoskeleton environmental perception and propose future directions.

Keywords: computer vision; environmental perception; gait planning; lower-limb exoskeleton robots.

Figure 1

The role of vision in the human–machine–environment loop of lower-limb exoskeletons.

Figure 2

Common visual sensors: (a) Philips’ RGB network camera [46]; (b) ZED’s binocular stereo vision camera, the ZED Mini Stereo Camera [47]; (c) Unitree’s LiDAR L1 [48]; (d) Realsense’s depth camera, the D435i [49]; (e) Realsense’s LiDAR camera, the L515 [45].

Figure 3

The role and relationship of controllers at different levels.

Figure 4

Stair-line detection methods based on traditional image processing.

Figure 5

Illustration of StairNet with RGB-D inputs and StairNet with RGB input and depth estimation.

Figure 6

Process of plane-based stair detection methods.

Figure 7

The main process of environment-oriented adaptive gait-planning methods, where $F(H,W,l_1,l_2,T)$ represents the fitted joint spatio-temporal domain equation, and H and W represent the width and height of the stairs, respectively. $l_1$ and $l_2$ represent the thigh length and calf length, respectively. T represents the gait period. ${\tau }^2\ddot{y}={\alpha }_y({\beta }_y(g-y)-\tau \dot{y})+f$ represents the basic formula of a DMP. y represents the system status, and $\dot{y}$ and $\ddot{y}$ represent the first and second derivatives of y, respectively. g represents the target status, ${\alpha }_y$ and ${\beta }_y$ are two constants, f is the forcing term, and $\tau$ is the scale factor.