Enhanced Surgeons: Understanding the Design of Augmented Reality Instructions for Keyhole Surgery

Abstract

It is important to understand how to design AR content for surgical contexts to mitigate the risk of distracting the surgeons. In this work, we test information overlays for AR guidance during keyhole surgery. We performed a preliminary evaluation of a prototype, focusing on the effects of colour, opacity, and information representation. Our work contributes insights into the design of AR guidance in surgery settings and a foundation for future research on visualisation design for surgical AR.

Keywords: Human-centered computing; Treemaps; Visualization; Visualization design and evaluation methods; Visualization techniques.

Figure 1

A: Study setup, a laparoscopy plastic torso with laparoscopy camera and scissors to simulate keyhole surgery. B: View inside the plastic torso with no augmented reality, showing the cardboard phantom with a marble in the cavity representing the tumour. During the experiment this view is covered by a neoprene sheet to represent the abdominal wall, so the participants can only see what is happening via the video screen. C: An example of one of the augmented reality overlays used, here a green wireframe to represent the liver, and a solid, low opacity, green tumour.

Figure 2

A: An example overlay using a blue wireframe model of the liver with a black plastic spider placed on the scene. The spider was used to test for inattentional blindness. B: An image of the experiment in progress (pixelation to maintain author anonymity). The user, at left, is attempting to perform a simulated surgical task on the white anatomical phantom to their right. The user is using a rigid laparoscopic tool, which consists of a pair of pincers actuated by the handle of the tool. The user cannot see directly into the phantom, but is instead using video images fed from the laparoscopic camera on their right and shown on the screen in front of them.