Object Manipulation with an Anthropomorphic Robotic Hand via Deep Reinforcement Learning with a Synergy Space of Natural Hand Poses

Abstract

Anthropomorphic robotic hands are designed to attain dexterous movements and flexibility much like human hands. Achieving human-like object manipulation remains a challenge especially due to the control complexity of the anthropomorphic robotic hand with a high degree of freedom. In this work, we propose a deep reinforcement learning (DRL) to train a policy using a synergy space for generating natural grasping and relocation of variously shaped objects using an anthropomorphic robotic hand. A synergy space is created using a continuous normalizing flow network with point clouds of haptic areas, representing natural hand poses obtained from human grasping demonstrations. The DRL policy accesses the synergistic representation and derives natural hand poses through a deep regressor for object grasping and relocation tasks. Our proposed synergy-based DRL achieves an average success rate of 88.38% for the object manipulation tasks, while the standard DRL without synergy space only achieves 50.66%. Qualitative results show the proposed synergy-based DRL policy produces human-like finger placements over the surface of each object including apple, banana, flashlight, camera, lightbulb, and hammer.

Keywords: anthropomorphic robotic hand; deep reinforcement learning; natural hand poses; object grasping; object relocation; synergy space.

Figure 1

The proposed synergy–based DRL for natural object manipulation with a synergy space of object haptic information and a deep regressor to derive natural hand poses of the ADROIT anthropomorphic robotic hand.

Figure 2

From human demonstrations (top row), haptic information is shown in point clouds (second row). CNF encodes the haptic information, $q_{\varphi }$ into a synergy space representation, $𝓏$. The deep regressor, $R_{\theta }\left(𝓏\right)$ estimates natural hand poses (final row) in time series.

Figure 3

Reconstruction of object haptic information using the synergy-based encoder-decoder via CNF for three synergy dimensions. The illustrations are derived from training iterations: (a), (c), and (e) point clouds shown in gray are the ground truth haptic maps, the (b), (d), and (f) point clouds in blue are the reconstructed maps from CNF models with dimensions of $z^{32},z^{16},z^4$, respectively.

Figure 4

The mean squared error (c) between the joint angle distributions of the haptic map dataset, (a) and the angles distributions estimated by the regressor (b) for each finger in the anthropomorphic hand. A lower error indicates higher confidence in estimating the joint angles of a natural grasping pose.

Figure 5

The left column shows the average sum of rewards from training the manipulation task with the proposed synergy-based DRL in purple and synergy-less standard DRL in blue. The solid line represents the mean and the shadow area of the standard deviation of the rewards. The right column shows the time-series frames from grasping and relocation of each object from apple, banana, hammer, lightbulb, flashlight, and camera with the standard DRL in the blue frames of (a,c,e,g,i,k), and the proposed synergy-based DRL in the purple frames of (b,d,f,h,j,l).