Visual Haptic Feedback for Training of Robotic Suturing

Abstract

Current surgical robotic systems are teleoperated and do not have force feedback. Considerable practice is required to learn how to use visual input such as tissue deformation upon contact as a substitute for tactile sense. Thus, unnecessarily high forces are observed in novices, prior to specific robotic training, and visual force feedback studies demonstrated reduction of applied forces. Simulation exercises with realistic suturing tasks can provide training outside the operating room. This paper presents contributions to realistic interactive suture simulation for training of suturing and knot-tying tasks commonly used in robotically-assisted surgery. To improve the realism of the simulation, we developed a global coordinate wire model with a new constraint development for the elongation. We demonstrated that a continuous modeling of the contacts avoids instabilities during knot tightening. Visual cues are additionally provided, based on the computation of mechanical forces or constraints, to support learning how to dose the forces. The results are integrated into a powerful system-agnostic simulator, and the comparison with equivalent tasks performed with the da Vinci Xi system confirms its realism.

Keywords: collision detection; haptics; knot-tying; minimally invasive surgery; robotic surgery training; surgical simulation; virtual reality.

FIGURE 1

The computation of the deformation is done in the local reference frame F of the beam element (in red). Using this frame, the 6D local displacements of the nodes are computed between the current configuration and their rest configuration (in orange).

FIGURE 2

Construction of the spline control points from the beam end nodes.

FIGURE 3

To construct R _F we first evaluate the tangent at the cubic Bézier curve mid point (A) We then transform each beam end point orientation to align its first axis with the mid point tangent (B) We finally compute the orientation of F by taking the average of these two rotations (C).

FIGURE 4

Alignment of $u_{x_i}$ with the mid point tangent.

FIGURE 5

Visual stress on a thread from floppy (A) to tight (B,C) condition until suture breakage with tension release (D).

FIGURE 6

We test four different discretizations of a suture thread of 12 cm subjected to gravity. In increasing order of discretization red—orange—blue—green. The dynamics of the pendular motion is captured at time 0[s] (A), 0.1[s] (B), 0.2[s] (C) and 0.3[s] (D). The model converges to a unique solution as the level of discretization increases.

FIGURE 7

We apply a torsional motion at the end points of the thread. The coupling between bending and torsion allows the formation of plectonemes.

FIGURE 8

A suture thread is wrapped around a fixed rigid cylinder by pulling at its ends. The wrapping motion is sampled at time 0 [s](A), 0.5 [s](B), and 1 [s](C). The model remains inextensible during this simulation, which is why the loops around the rigid cylinder get closer and closer.

FIGURE 9

Gradual tightening of a simple knot. The motion is sampled at time 0 [s](A), 1 [s](B), and 2 [s](C). The accurate contact model allows to form the knot and to tighten it.

FIGURE 10

First step of a surgeon’s knot. The double overhand knot is created by wrapping the thread around an instrument tip (A), grasping the other end, and pulling it through the double loop, resulting in a pattern with two twists (B). The knot position can be adjusted by exerting traction on one end, thus moving the knot over to the other end (C,D). The collision is sufficiently robust to support thread interactions with itself and with the other elements.

FIGURE 11

Simulation accurately reproducing knot-tying errors. An accidental loop is created when one end of the thread is not completely pulled through the loop before tightening the knot on the rings (A). An air knot is created when the knot is not properly guided down onto the rings before tightening it with traction on both ends of the thread (B).

FIGURE 12

Interrupted suturing (A,B) and continuous suturing (C,D).

FIGURE 13

Visual stress gradients on a thread when tightening a continuous suture (A), and on tissue when driving a needle through tissue (right instrument) while applying a counter-pressure (left instrument) (B). Combined visual cues on thread and tissue when applying excessive force (C) leading to a tear in the tissue and subsequently reduced tension on the thread (D).

FIGURE 14

Visual cues for excessive forces during collisions: Visual cues for excessive forces during collisions. Instrument tips turning from silver to orange during collision when forming a loop with a thread for knot-tying (A), right instrument tip turning orange during collision with the dome, which turns from blue to pink (B), double grasping with shearing forces during needle manipulation with both instrument tips turning orange, and the needle approximating the breakage point as represented by its color change to red (C). Animated scenes are shown in the Supplementary Video S1.

FIGURE 15

Task performance comparison between a real robotic system and the simulation. The knot-tying sequence is based on a surgeon’s knot. The task with the da Vinci Xi system is shown on the left, and the simulated task is shown without (middle) and with visual haptic cues (right).