Integrating Eye-Tracking to Augmented Reality System for Surgical Training

Abstract

Augmented Reality has been utilized for surgical training. During the implementation, displaying instructional information at the right moment is critical for skill acquisition. We built a new surgical training platform combining augmented reality system (HoloLens, Microsoft) with an eye-tracker (Pupil labs, Germany). Our goal is to detect the moments of performance difficulty using the integrated eye-tracker so that the system could display instructions at the precise moment when the user is seeking instructional information during a surgical skill practice in simulation. In the paper, we describe the system design, system calibration and data transferring between these devices.

Keywords: Augmented reality; Calibration; Eye tracking; Simulation; Surgical training.