Sensing expectation enables simultaneous proprioception and contact detection in an intelligent soft continuum robot

Abstract

A high-level perceptual model found in the human brain is essential to guide robotic control when facing perception-intensive interactive tasks. Soft robots with inherent softness may benefit from such mechanisms when interacting with their surroundings. Here, we propose an expected-actual perception-action loop and demonstrate the model on a sensorized soft continuum robot. By sensing and matching expected and actual shape (1.4% estimation error on average), at each perception loop, our robot system rapidly (detection within 0.4 s) and robustly detects contact and distinguishes deformation sources, whether external and internal actions are applied separately or simultaneously. We also show that our soft arm can accurately perceive contact direction in both static and dynamic configurations (error below 10°), even in interactive environments without vision. The potential of our method are demonstrated in two experimental scenarios: learning to autonomously navigate by touching the walls, and teaching and repeating desired configurations of position and force through interaction with human operators.

Fig. 1. An expected-actual comparison perception-action loop in soft robots.

A Overview of the sensorized rod-driven soft continuum robot (RDSR) and the perception method. B A general framework for high-level perception-action loops. C Demonstrations of perception-intensive environmental interaction tasks: autonomous exploration and navigation of mazes by detecting contacts with the walls, and teaching and repeating tasks by interacting with human operators.

Fig. 2. Detailed scheme of expected-actual comparison method on an RDSR and the performance of soft sensors.

A Proprioception model of the sensorized RDSR. B Detailed implementation of the perception-action loop in the sensorized RDSR. C Real-time changes in sensors and the perceived AS. Cases: (1) only external contact applied, (2) only actuation applied, (3) external contact applied after actuation. The real-shape image and perceived AS under case (1) are presented in (i, ii). The results under cases (2) and (3) are shown as (iii, iv, v). All sensor data are presented in (vi). D Tip position errors between perception and camera under different loading, from 100 to 700 gf in 100 gf intervals. E Data distribution of the axial and norm distance between AS and ES under no-loading conditions. The data are on the left of the box. Each box contains the data scale, median, and mean value.

Fig. 3. Proprioception and simultaneous contact detection during motion.

Given the different sets of continuous control commands, the robot is blocked by obstacles and actively perceives (A) tip contact detection and (B) sensor contact detection. C The experiment on contact with material of varying softness, including Ecoflex10 (Smooth-On, Inc. 55 kPa), Ecoflex30 (69 kPa), Dragonskin10 (151 kPa), Dragonskin30 (593 kPa), and printed PLA (Rigid).

Fig. 4. Perception of contact direction, force, and shape.

The capabilities in perceiving contact direction and force in static (A) and dynamic (B) configurations. C Detection and perception experiments under ink. (i) Experimental setup. (ii) The image of a real shape under the ink. (iii) Detected shapes with and without the proposed EP method. (iv) Contact index ε_x and ε_y along x- and y-axes. (v) Perceived contact force profiles and vectors.

Fig. 5. Soft robots automatically explore and navigate a circular maze.

A Photograph of the experimental setup for maze exploration. The maze is designed with multiple intersections and barrier walls. The end-effector of the soft robot is inserted into the channel to touch the wall and find a pathway. B Top view of the successful right path and its relative position in the maze. Each right waypoint is determined from the exploration process. The robot will record and repeat them after finishing the process. C Tip coordinates and the corresponding norm, x, and y-axis value of ε when the robot moves along the right path. D The norm, x, and y-axis value of ε throughout the automatic exploration process. E Partial enlarged view of the norm, x and y-axis value of ε from 130 to 200 s. F Display of the whole exploration process including updated target point, perceived ES and AS position, and their relative location in the maze. G Partial enlarged view of the exploration process from 130 to 200 s.

Fig. 6. Teaching and repeating test on a manikin.

The human operator teaches the robot to move to the desired configuration of position and force. Then, the robot repeats them. A Experimental setup for manikins and soft robots. (i) Overview of the setup. The manikin is lying horizontally, and the soft robot is mounted on top of it. (ii) Top view of the manikin with desired positions. The desired points are marked in green. The robot moves sequentially from p₁ to p₁₀. (iii) A detailed setup image of the sensors controlling and measuring the desired force. A uniaxial force sensor is mounted vertically to measure the pushing force between the manikin and the robot. Two soft sensors R_e and R_s are attached near the tip to control the magnitude of the force. B Teaching the robot to move to the marked position in the manikin and apply the desired pushing force to it. (i) the real-time sensor data including force, R_e, and R_s sensors during the teaching period. (ii-iii) the sensor data and robot configurations at moment of t₁ and t₂. C The tip trajectories in the x-y plane during teaching and repeating. D Tip position errors at all target markers after repeating. E The repeated and taught force at each marker.