Learning feature descriptors for pre- and intra-operative point cloud matching for laparoscopic liver registration

Abstract

Purpose: In laparoscopic liver surgery, preoperative information can be overlaid onto the intra-operative scene by registering a 3D preoperative model to the intra-operative partial surface reconstructed from the laparoscopic video. To assist with this task, we explore the use of learning-based feature descriptors, which, to our best knowledge, have not been explored for use in laparoscopic liver registration. Furthermore, a dataset to train and evaluate the use of learning-based descriptors does not exist.

Methods: We present the LiverMatch dataset consisting of 16 preoperative models and their simulated intra-operative 3D surfaces. We also propose the LiverMatch network designed for this task, which outputs per-point feature descriptors, visibility scores, and matched points.

Results: We compare the proposed LiverMatch network with a network closest to LiverMatch and a histogram-based 3D descriptor on the testing split of the LiverMatch dataset, which includes two unseen preoperative models and 1400 intra-operative surfaces. Results suggest that our LiverMatch network can predict more accurate and dense matches than the other two methods and can be seamlessly integrated with a RANSAC-ICP-based registration algorithm to achieve an accurate initial alignment.

Conclusion: The use of learning-based feature descriptors in laparoscopic liver registration (LLR) is promising, as it can help achieve an accurate initial rigid alignment, which, in turn, serves as an initialization for subsequent non-rigid registration.

Keywords: 3D feature descriptors; Laparoscopic liver registration; Laparoscopic liver surgery; Non-rigid registration; Point cloud matching.

Fig. 1

Schematic description of the generation of the source $(S)$ and target $(T)$ point clouds based on 16 liver surface models from the 3D-IRCADb-01 dataset.

Fig. 2

LiverMatch network overview: 1) Encoder - down-samples input point clouds and extract associated features; 2) Transformer - updates features to conditioned features with self-global geometry and cross-global geometry information. 3) Decoder - up-samples conditioned features to obtain per-point features. 4) Matching - calculates a confidence matrix to select matches. An additional 1D convolution decodes the ${\mathbf{x}}^{\mathbf{S}}$ to visibility scores.

Fig. 3

Visualization of matches and registration results of FPFH, Predator, and LiverMatch on two pairs of the source point cloud (blue) and target point cloud (red). The first two rows show the results for the same pair of source and target point clouds, while the last two show the results for another pair of source and target point clouds. Unmatched points are shown in gray (the first and third rows). Note that point clouds in FPFH are down-sampled.