Beyond the visible: preliminary evaluation of the first wearable augmented reality assistance system for pancreatic surgery

Abstract

Purpose: The retroperitoneal nature of the pancreas, marked by minimal intraoperative organ shifts and deformations, makes augmented reality (AR)-based systems highly promising for pancreatic surgery. This study presents preliminary data from a prospective study aiming to develop the first wearable AR assistance system, ARAS, for pancreatic surgery and evaluating its usability, accuracy, and effectiveness in enhancing the perioperative outcomes of patients.

Methods: We developed ARAS as a two-phase system for a wearable AR device to aid surgeons in planning and operation. This system was used to visualize and register patient-specific 3D anatomical models during the surgery. The location and precision of the registered 3D anatomy were evaluated by assessing the arterial pulse and employing Doppler and duplex ultrasonography. The usability, accuracy, and effectiveness of ARAS were assessed using a five-point Likert scale questionnaire.

Results: Perioperative outcomes of five patients underwent various pancreatic resections with ARAS are presented. Surgeons rated ARAS as excellent for preoperative planning. All structures were accurately identified without any noteworthy errors. Only tumor identification decreased after the preparation phase, especially in patients who underwent pancreaticoduodenectomy because of the extensive mobilization of peripancreatic structures. No perioperative complications related to ARAS were observed.

Conclusions: ARAS shows promise in enhancing surgical precision during pancreatic procedures. Its efficacy in preoperative planning and intraoperative vascular identification positions it as a valuable tool for pancreatic surgery and a potential educational resource for future surgical residents.

Keywords: Augmented reality; Navigation system; Pancreatectomy; Surgical planning; Wearable devices.

Fig. 1

a Preparation of the surgical field includes the fixation of a sterilized marker onto the xiphoid process before the laparotomy and b marker-based tracking using the entry of left renal vein into the inferior vena cava as the registration point

Fig. 2

Intraoperative evaluation of the location and precision of the registered 3D anatomy using ultrasonography. Portal vein, PV; splenic vein, SV; superior mesenteric vein, SMV

Fig. 3

a Borderline resectable pancreatic body cancer (*), infiltrating the celiac trunk (CT) and common hepatic artery (CHA); b CHA tumor infiltration (*) and blood supply via gastroduodenal artery (GDA); c the figure illustrating angiographic closure of the CHA and reversal blood supply of hepatic artery (HA) via superior mesenteric artery (SMA) and GDA

Fig. 4

a Reconstructed 3D model of tumor with peripancreatic vascular system; b illustration of planned procedure as extended distal pancreatosplenectomy with an en-bloc resection of the common hepatic artery (CHA) and celiac axis; c intraoperative snapshot during vascular preparation before resection phase using ARAS. Exposition and marking the CHA (red vascular loop). Gastroduodenal artery, GDA, splenic artery, SA; hepatic artery, HA; portal vein, PV; superior mesenteric artery, SMA; left gastric artery, LGA

Fig. 5

Preoperative planning of the procedure using a computed tomography images and b 3D model, illustrating the tortuous course of the splenic artery (SA), and proximity of the tumor (*) and splenic vein (SV). Portal vein, PV; inferior mesenteric vein, IMV

Fig. 6

Intraoperative snapshot illustrating a the preparation phase using ARAS, b identification of vascular system using ARAS after resection phase; c resected specimen. Splenic vein, SV; splenic artery, SA; portal vein, PV; hepatic artery, HA; superior mesenteric vein, IMV; inferior mesenteric vein, IMV

Fig. 7

Intraoperative snapshot illustrating a the preparation of the gastroduodenal artery (GDA) and portal vein (PV) using ARAS, b identification of vascular system and dorsal tumor infiltration into the PV (*) after pancreas transection, and c PV reconstruction after resection phase. Common hepatic artery, CHA; splenic vein, SV; superior mesenteric artery, SMA; superior mesenteric vein, SMV

Fig. 8

a Responses on five-point Likert scale questionnaire (in percent; the figures indicate the frequency per response category; the response categories range from “very poor” (1) to “excellent” (5). b The box plot illustrating the assessment of usability, accuracy, and overall effectiveness of ARAS for pre- and intraoperative sessions. c Illustration of usability, accuracy, and overall effectiveness of ARAS between two different surgical procedures. Planning phase: Q1–2; preresection phase: Q3–6; resection phase: Q 7–8. Questions 1–8 were also presented in Table 1. Pylorus preserving pancreaticoduodenectomy, PPPD