"Hey Siri! Perform a type 3 hysterectomy. Please watch out for the ureter!" What is autonomous surgery and what are the latest developments?

Abstract

As a result of major advances in deep learning algorithms and computer processing power, there have been important developments in the fields of medicine and robotics. Although fully autonomous surgery systems where human impact will be minimized are still a long way off, systems with partial autonomy have gradually entered clinical use. In this review, articles on autonomous surgery classified and summarized, with the aim of informing the reader about questions such as "What is autonomic surgery?" and in which areas studies are progressing.

Keywords: Autonomous surgery; machine learning; robotic surgery; skill analysis; skill learning.

Figure 1

An image section taken during robotic surgery: a) Original image, b) binary segmentation (instruments blue, tissue red), c) multi-fold segmentation (three separate regions of the instrument can be identified: body, articulation joint, grasper tip), d) multiclass segmentation (each instrument can be defined separately) (5)

Figure 2

Steps in the process of grasping a surgical needle stuck into a tissue phantom at an appropriate angle and extracting it from the tissue by calculating the tensile force vector. a) Calibration, b) snapshot image creation, c) planning the segmentation and extraction phase, d) approaching phase, e) withdrawal phase, f) complete removal of the needle from the tissue (10)

Figure 3

The segmentation regions (tail, body and tip) determined by the tracking algorithm that adapts to changing positions autonomously. a) Ideal position, b) position at the border of the endoscope’s visual field, c) the position at which the needle angle changes in the spatial plane. Segmentation of the needle was successful in all three cases (11)

Figure 4

Fully autonomous suturing (needle grasping, needle transfer between instruments, suturing and knot tying) performed on a gel phantom after determining the starting point, distance between sutures and tissue depth (12)

Figure 5

Identification of the suture thread along the entire length by segmentation and additionally determining the control points for motion tracking (tracking the dynamic relationship of the changing points in respect to each other is especially important during the knot tying phase). a) Multiple control points along the suture length, b) as a knot is tightened, the control points get closer and closer, c) after sufficient knot tightening, the knot with multiple points is redefined after a while, d) as a single control point (13)

Figure 6

STAR procedures; a) intestinal anastomosis in ex-vivo pig tissue, b) in-vivo vaginal cuff (RGB camera view) after hysterectomy of pig, c) plenoptic camera view with the markings of the beginning, end and suture transition points (vaginal cuff closure was completed in about five minutes) (14) STAR: Smart Tissue Anastomosis Robot

Figure 7

Trajectory of rolling arc looping (15)

Figure 8

Autonomously calculated traction and counter-traction to overcome the problem of significant tissue deformation. a) Directing the biopsy needle tip towards the target, b) adjusting the needle exit point, c) completing the circular cutting process with high precision (21)

Figure 9

a) Suturing, b) needle passing, c) knotting tasks (parameters evaluated and reported by JIGSAWS) (22)

Figure 10

A machine learning model in which surgical performance in the surgical treatment of prostate cancer is measured with OPM and its effects on hospital stay are predicted (24) OPM: Automated performance metrics