Data-Driven Shape Sensing of a Surgical Continuum Manipulator Using an Uncalibrated Fiber Bragg Grating Sensor

Abstract

This article proposes a data-driven learning-based approach for shape sensing and Distal-end Position Estimation (DPE) of a surgical Continuum Manipulator (CM) in constrained environments using Fiber Bragg Grating (FBG) sensors. The proposed approach uses only the sensory data from an unmodeled uncalibrated sensor embedded in the CM to estimate the shape and DPE. It serves as an alternate to the conventional mechanics-based sensor-model-dependent approach which relies on several sensor and CM geometrical assumptions. Unlike the conventional approach where the shape is reconstructed from proximal to distal end of the device, we propose a reversed approach where the distal-end position is estimated first and given this information, shape is then reconstructed from distal to proximal end. The proposed methodology yields more accurate DPE by avoiding accumulation of integration errors in conventional approaches. We study three data-driven models, namely a linear regression model, a Deep Neural Network (DNN), and a Temporal Neural Network (TNN) and compare DPE and shape reconstruction results. Additionally, we test both approaches (data-driven and model-dependent) against internal and external disturbances to the CM and its environment such as incorporation of flexible medical instruments into the CM and contacts with obstacles in taskspace. Using the data-driven (DNN) and model-dependent approaches, the following max absolute errors are observed for DPE: 0.78 mm and 2.45 mm in free bending motion, 0.11 mm and 3.20 mm with flexible instruments, and 1.22 mm and 3.19 mm with taskspace obstacles, indicating superior performance of the proposed data-driven approach compared to the conventional approaches.

Keywords: Continuum Manipulator; Data-driven Sensing; Deep Neural Networks; Fiber Bragg Grating; Shape Sensing; Temporal Neural Networks.

Fig. 1.

(a) Robot-assisted treatment of pelvic osteolysis, (b) continuum manipulator interacting with bone behind the acetbular implant, and (c) debriding tool and FBG sensor integrated with the continuum manipulator (distal end view)

Fig. 2.

(a) CM tip cross sectional view showing the actuation, sensing, and tool channels, (b) tool integrated with the CM, (c) FBG sensor, (d) FBG sensor with the triangular configuration cross section view, and (e) Top: DNN architecture. λ_n is the raw FBG vector at the n^th observation. ${\widehat{p}}_n$ is the network output CM tip position. Hyperparameters of the fully connected layers are listed under each block. Bottom: TNN architecture. The concatenation process is illustrated with the time-series data.

Fig. 3.

The experimental setup including the CM actuation unit integrated with a calibrated stereo camera pair. Raw images obtained from the camera pair are first color-segmented and then 3-D locations of the markers are computed by triangulation. The custom-designed jig for validation of the marker-based triangulation is shown on the bottom left.

Fig. 4.

(a through h) CM bending experiments in constrained environment in presence of internal disturbance (flexible tool) and external disturbance (obstacles at various locations), (i) CM bending in free environment, (j) CM bending in free environment with tool, as well as the demonstration of the obstacle locations relative to the CM for experiments a through h. (k) loss trend during training and validation.

Fig. 5.

Shape reconstruction results using the DNN, TNN, linear, and physics-based methods for the constrained environment experiments with obstacle contacts (a and d), free environment experiment with tool (b and e), free environment experiment without tool (c and f).