NrtNet: An Unsupervised Method for 3D Non-Rigid Point Cloud Registration Based on Transformer

Abstract

Self-attention networks have revolutionized the field of natural language processing and have also made impressive progress in image analysis tasks. Corrnet3D proposes the idea of first obtaining the point cloud correspondence in point cloud registration. Inspired by these successes, we propose an unsupervised network for non-rigid point cloud registration, namely NrtNet, which is the first network using a transformer for unsupervised large deformation non-rigid point cloud registration. Specifically, NrtNet consists of a feature extraction module, a correspondence matrix generation module, and a reconstruction module. Feeding a pair of point clouds, our model first learns the point-by-point features and feeds them to the transformer-based correspondence matrix generation module, which utilizes the transformer to learn the correspondence probability between pairs of point sets, and then the correspondence probability matrix conducts normalization to obtain the correct point set corresponding matrix. We then permute the point clouds and learn the relative drift of the point pairs to reconstruct the point clouds for registration. Extensive experiments on synthetic and real datasets of non-rigid 3D shapes show that NrtNet outperforms state-of-the-art methods, including methods that use grids as input and methods that directly compute point drift.

Keywords: NrtNet; non-rigid point cloud; registration; self-attentive; transformer; unsupervised.

Figure 1

The idea of our proposed NrtNet. The point cloud is first rearranged using transformer, and then the permuted point cloud is reconstructed to achieve the registration.

Figure 2

NrtNet is an unsupervised, end-to-end network for non-rigid point cloud registration. The source point cloud $A\in {\mathbb{R}}^{n\times 3}$ and the target point cloud $B\in {\mathbb{R}}^{n\times 3}$ are fed into the feature extraction module and the transformer module to generate the point set correspondence matrix $P\in {\mathbb{R}}^{n\times n}$. Then, the permuted point cloud is fed into the reconstruction module to generate the exact same point cloud $A_{last}$ as B, which achieves the purpose of registration.

Figure 3

Transformer module. The probability matrix $P_{rand}$ can be obtained by feeding the high-dimensional features of the point cloud $F_a\in {\mathbb{R}}^{n\times d}$ and $F_b\in {\mathbb{R}}^{n\times d}$ into the transformer encoder and transformer decoder, respectively. Then, the probability matrix $P_{rand}$ is fed into the smooth module to obtain the inverted exact correspondence matrix P.

Figure 4

Transformer Encoder. $\phi$, $\psi$ is a linear layer, $\alpha$ is an mlp and they are all feature transform layers, $\delta$ is a linear layer which is a position encoder and $\gamma$ is a mapping function.

Figure 5

Transformer decoder and the smooth module. These two modules convert the transformed features obtained from the transformer encoder into an exact point set correspondence matrix.

Figure 6

The reconstruction module concatenates the global features $v_b$ of the target point cloud to each permuted point cloud $A_{re\underset{\_}{}order}$, and feeds them into the MLP to reconstruct the point cloud $A_{last}$.

Figure 7

Quantitative comparison of point set correspondence rates for non-rigid registration under different methods.

Figure 8

In a qualitative comparison between Nrtnet and other methods in a large deformed human pose, the experiments show the effectiveness of different methods for non-rigid point cloud registration.

Figure 9

Quantitative comparison of point set correspondence rates for rigid registration under different methods.

Figure 10

Registration performance of Nrtnet in small deformation dataset paper, cloth, sweater, and t-shirt.

Figure 11

The point set correspondence rate under different losses. The experiments qualitatively compare the correspondence differences between NrtNet’s losses and ordinary losses.

Figure 12

The registration effect of NrtNet on the real scan data.