A Differentiable Extended Kalman Filter for Object Tracking Under Sliding Regime

Abstract

Tactile sensing represents a valuable source of information in robotics for perception of the state of objects and their properties. Modern soft tactile sensors allow perceiving orthogonal forces and, in some cases, relative motions along the surface of the object. Detecting and measuring this kind of lateral motion is fundamental to react to possibly uncontrolled slipping and sliding of the object being manipulated. Object slip detection and prediction have been extensively studied in the robotic community leading to solutions with good accuracy and suitable for closed-loop grip stabilization. However, algorithms for object perception, such as in-hand object pose estimation and tracking algorithms, often assume no relative motion between the object and the hand and rarely consider the problem of tracking the pose of the object subjected to slipping and sliding motions. In this work, we propose a differentiable Extended Kalman filter that can be trained to track the position and the velocity of an object under translational sliding regime from tactile observations alone. Experiments with several objects, carried out on the iCub humanoid robot platform, show that the proposed approach allows achieving an average position tracking error in the order of 0.6 cm, and that the provided estimate of the object state can be used to take control decisions using tactile feedback alone. A video of the experiments is available as Supplementary Material.

Keywords: differentiable extended kalman filtering; humanoid robotics; machine learning-aided filtering; object position tracking; object velocity tracking.

FIGURE 1

In (A), the exploded view of the sensing module. In (B), the CAD model of the fingertip adapter for the iCub finger hosting two sensing modules. In (C), the final assembled fingertip adapters with the grip tape cover. In (D), the experimental setup with the iCub humanoid robot left hand equipped with uSkin deformable soft tactile sensors.

FIGURE 2

Overview of the proposed differentiable filtering architecture for object sliding tracking. σ indicates the ReLU activation function.

FIGURE 3

Comparison between the desired grasp strength and the achieved grasp strength for the index and middle fingers during the execution of a controlled sliding experiment.

FIGURE 4

Outcome of the controlled sliding experiment with the box-shaped object.

FIGURE 5

In (A), comparison between the ground truth position from the ArUco marker detection system $p_t^A$, its filtered version $p_{t|t}^{sm}$ and its smoothed version $p_{t|T}^{sm}$ obtained using a Kalman filter and smoother respectively. In (B), comparison between the ArUco velocity signal obtained using finite differences, filtered and smoothed version.

FIGURE 6

Picture of the objects used in the experiments. From the left, the box-shaped object, the water bottle and the mustard bottle.

FIGURE 7

Data traces for one of the data collection experiment performed using the mustard bottle. The tactile signals correspond to the raw sensor reading using arbitrary units.

FIGURE 8

Comparison of the position and velocity trajectories for several configuration of the tactile measurements with the ground truth state for the object mustard bottle.

FIGURE 9

Sample trajectories from one of the experiments on using the learned filter in a practical scenario with the box-shaped object. In (A), the results are shown when using the xy tactile channels, in (B) when using the xyz tactile channels.