Low cost exoskeleton manipulator using bidirectional triboelectric sensors enhanced multiple degree of freedom sensory system

Abstract

Rapid developments of robotics and virtual reality technology are raising the requirements of more advanced human-machine interfaces for achieving efficient parallel control. Exoskeleton as an assistive wearable device, usually requires a huge cost and complex data processing to track the multi-dimensional human motions. Alternatively, we propose a triboelectric bi-directional sensor as a universal and cost-effective solution to a customized exoskeleton for monitoring all of the movable joints of the human upper limbs with low power consumption. The corresponding movements, including two DOF rotations of the shoulder, twisting of the wrist, and the bending motions, are detected and utilized for controlling the virtual character and the robotic arm in real-time. Owing to the structural consistency between the exoskeleton and the human body, further kinetic analysis offers additional physical parameters without introducing other types of sensors. This exoskeleton sensory system shows a great potential of being an economic and advanced human-machine interface for supporting the manipulation in both real and virtual worlds, including robotic automation, healthcare, and training applications.

Fig. 1. Triboelectric bidirectional (TBD) sensor-integrated exoskeleton system.

a Schematics of exoskeleton sensory system for realizing the manipulation in virtual space and robotics. b Working principle of TBD sensor. c Key functions for achieving the multidimensional motion sensing of the upper limb, (i) basic bidirectional rotation sensing, (ii) bidirectional rotation sensing with two degree of freedoms (DOFs), (iii) bidirectional detection of twisting motion of the wrist, and (iv) bidirectional linear motion sensing for finger bending. Photo credit: Minglu Zhu, National University of Singapore.

Fig. 2. Characterization and optimization of triboelectric bidirectional (TBD) sensors.

a Configuration of the assembly of the exoskeleton sensory system, with the rotational triboelectric bidirectional back (RTBD-B) sensor, the rotational triboelectric bidirectional shoulder (RTBD-S) sensor, the rotational triboelectric bidirectional elbow (RTBD-E) sensor, the rotational triboelectric bidirectional wrist (RTBD-W) sensor, and the linear triboelectric bidirectional finger (LTBD-F) sensor. b (i) Configuration of the varied grating widths (1, 3, 5, and 7 mm) with a constant spacing of 3 mm for a rotation TBD (RTBD) sensor, (ii) measurement of triboelectric output signals from the rotation speed of 10 revolutions per minute (RPM) to 300 RPM, and (iii) the enlarged waveforms of 10 RPM and 300 RPM. c (i) Configuration of the varied spacing (1, 2, and 3 mm) with a constant width of 3 mm for a RTBD sensor, (ii) measurement of triboelectric output signals from the rotation speed of 10 to 300 RPM, and (iii) the enlarged waveforms of 10 RPM and 200 RPM. d Configuration and measured triboelectric output signals for a linear TBD (LTBD) sensor with the varied spacing during the finger bending of 90°. e Response latency of switch (1.5 mm gap) during the changing of the rotation directions. The inserted graph is the enlarged waveform of 10 RPM, and the time of the peak voltages of the separation (1) and contact (2) signals are provided. f Measured signals for bidirectional rotation with the grating pattern of four varied widths (1, 3, 5, and 7 mm).

Fig. 3. Signal processing and demonstration in virtual space.

a Flow chart of triboelectric signal processing for manipulation in virtual space. b Examples of the original triboelectric signals after the preprocessing circuit. c Demonstration of controlling virtual character: (i) the controllable motions and the activated sensors with the limits of motion ranges for the rotational triboelectric bidirectional back (RTBD-B) sensor, the rotational triboelectric bidirectional shoulder (RTBD-S) sensor, the rotational triboelectric bidirectional elbow (RTBD-E) sensor, the rotational triboelectric bidirectional wrist (RTBD-W) sensor, and the linear triboelectric bidirectional finger (LTBD-F) sensor and (ii) the real-time signals during the manipulation, the output channels of BF, BB, SF, SB, EF, EB, WF, WB, FF, and FB stand for the forward (clockwise) and the backward (counterclockwise) rotations of the RTBD-B sensor, the RTBD-S sensor, the RTBD-E sensor, the RTBD-W sensor, and the LTBD-F sensor, i.e., BF for forward rotation of RTBD-B. Photo credit: Minglu Zhu, National University of Singapore.

Fig. 4. Signal processing and demonstration in robotic control.

a Flow chart of triboelectric signal processing for manipulation of robotic arms, and the photograph of the exoskeleton sensory system (right arm) with the rotational triboelectric bidirectional back (RTBD-B) sensor, the rotational triboelectric bidirectional shoulder (RTBD-S) sensor, the rotational triboelectric bidirectional elbow (RTBD-E) sensor, the rotational triboelectric bidirectional wrist (RTBD-W) sensor, and the linear triboelectric bidirectional finger (LTBD-F) sensor. b Demonstration of joint works of two robotic arms for picking up the cube and placing it into the box, (i) flow chart of the motions, and the activated arm and the sensors. (ii) the real-time signals during the manipulation, the channels of BF, BB, SF, SB, EF, EB, WF, WB, FF, and FB represent the forward (clockwise) and the backward (counterclockwise) rotations of the RTBD-B sensor, the RTBD-S sensor, the RTBD-E sensor, the RTBD-W sensor, and the LTBD-F sensor. Photo credit: Minglu Zhu, National University of Singapore.

Fig. 5. Demonstration of ping-pong training program for simultaneous monitoring of multiple sensing signals.

a Illustrations of four striking motions in ping-pong game, (i) forehand stroke, (ii) left sidespin, (iii) right sidespin, and (iv) smash. b The main activated sensors on exoskeleton for the corresponding strikes, the rotation direction of each sensor is marked by red arrow. c The real-time signals generated from four strikes, the channels of BF, BB, SF, SB, EF, EB, WF, WB, FF, and FB represent the forward (clockwise) and the backward (counterclockwise) rotations of the rotational triboelectric bidirectional back (RTBD-B) sensor, the rotational triboelectric bidirectional shoulder (RTBD-S) sensor, the rotational triboelectric bidirectional elbow (RTBD-E) sensor, the rotational triboelectric bidirectional wrist (RTBD-W) sensor, and the linear triboelectric bidirectional finger (LTBD-F) sensor.

Fig. 6. Force estimation with kinetic analysis of sensory information from rotational triboelectric bidirectional (RTBD) sensor.

a (i) Schematics and (ii) the generated signals from the rotational triboelectric bidirectional shoulder (RTBD-S) sensor and the rotational triboelectric bidirectional elbow (RTBD-E) sensor during the punch, the channels of BF, BB, SF, SB, EF, EB, WF, WB, FF, and FB represent the forward (clockwise) and the backward (counterclockwise) rotations of the rotational triboelectric bidirectional back (RTBD-B) sensor, the rotational triboelectric bidirectional shoulder (RTBD-S) sensor, the rotational triboelectric bidirectional elbow (RTBD-E) sensor, the rotational triboelectric bidirectional wrist (RTBD-W) sensor, and the linear triboelectric bidirectional finger (LTBD-F) sensor. L1 and L2 represent the length of the upper arm and the forearm, α, β, and γ represent the angles between the upper arm and the centerline, the forearm and the centerline, as well as the forearm and the upper arm. b (i) Schematics and (ii) measured data for the relationship between three angles and the position of the fist. c Comparison between the measured position and the position calculated by the sensory information. d The estimated instantaneous linear velocity of the fist against the varied angle of α for the given rotation speeds of 10, 50, 100 and 200 revolutions per minute (RPM). e The estimated punching force exerted on the target against the changing velocity of the fist and the variation of the duration for stopping, the mass of the participant’s arm is ~5.5 kg. f The estimated punching force exerted on the target against the varied angle of α for a specific rotation speed, the actual effective motion range of the fist is shaded. g Demonstration of punching force estimation through the data of the rotation sensing from the RTBD sensors in virtual space, (i) process of determining the level of the virtual punch based on the rotation sensing information, (ii) light punch and heavy punch demonstrations. Photo credit: Minglu Zhu, National University of Singapore.