Automatic patient positioning based on robot rotational workspace for extended reality

Abstract

Purpose: Understanding the properties and aspects of the robotic system is essential to a successful medical intervention, as different capabilities and limits characterize each. Robot positioning is a crucial step in the surgical setup that ensures proper reachability to the desired port locations and facilitates docking procedures. This very demanding task requires much experience to master, especially with multiple trocars, increasing the barrier of entry for surgeons in training.

Methods: Previously, we demonstrated an Augmented Reality-based system to visualize the rotational workspace of the robotic system and proved it helps the surgical staff to optimize patient positioning for single-port interventions. In this work, we implemented a new algorithm to allow for an automatic, real-time robotic arm positioning for multiple ports.

Results: Our system, based on the rotational workspace data of the robotic arm and the set of trocar locations, can calculate the optimal position of the robotic arm in milliseconds for the positional and in seconds for the rotational workspace in virtual and augmented reality setups.

Conclusions: Following the previous work, we extended our system to support multiple ports to cover a broader range of surgical procedures and introduced the automatic positioning component. Our solution can decrease the surgical setup time and eliminate the need to repositioning the robot mid-procedure and is suitable both for the preoperative planning step using VR and in the operating room-running on an AR headset.

Keywords: Augmented reality; Computer-assisted surgery; Patient positioning.

Fig. 1

Three representations of the reachability map of the robot. The 5D map can be shown as a set of spheres, with the red patches indicating the reachable orientation of the endoscope while the tip is at the center (left). The dimensionality-reduced reachability map is shown overlaid on a plane (center) and on overlaid on the volumetric rendering of the preoperative patient data (right)

Fig. 2

A photograph taken through the right lens of Magic Leap 1 headset. It shows a patient’s mesh model with 4 defined trocar positions and a calculated optimal location of the robotic arm. The robotic arm was set to a random joint configuration

Fig. 3

Graphical representation of the rotational sphere showing reachable orientations (red patches) at a given location. The green circle is indicating the maximal range of motion possible in any direction in a circular pattern; it is evaluated by finding the largest circle that can be drawn about the desired trocar direction (indicated by the blue line). The angle alpha describes the apex angle of the resulting cone

Fig. 4

Three consecutive steps of our automatic positioning algorithm showcase the improvement in the rotational workspace at the trocar location for a single-port cholecystectomy. The patient’s/robot’s model is moved or rotated incrementally with each iteration to reach the optimal pose. Each section displays the abdominal cavity with a blue target (gallbladder), the rotational sphere (refer to Fig. 3), and a green semitransparent cone, representing the angular range of motion within the patient’s body

Fig. 5

Calculation times of our algorithms based on the surface coverage metric (left) and cone angle (right) for two devices: Desktop PC (blue) and Magic Leap 1 (pink). Data presented for 4 different trocar configurations. Less is better.

Fig. 6

A comparison between running our algorithms for the first time (green) and in consecutive runs (yellow) on Magic Leap 1 AR headset. Left side: optimal pose calculation based on surface coverage; right side: based on cone angle. Less is better.