Quantifying attention shifts in augmented reality image-guided neurosurgery

Abstract

Image-guided surgery (IGS) has allowed for more minimally invasive procedures, leading to better patient outcomes, reduced risk of infection, less pain, shorter hospital stays and faster recoveries. One drawback that has emerged with IGS is that the surgeon must shift their attention from the patient to the monitor for guidance. Yet both cognitive and motor tasks are negatively affected with attention shifts. Augmented reality (AR), which merges the realworld surgical scene with preoperative virtual patient images and plans, has been proposed as a solution to this drawback. In this work, we studied the impact of two different types of AR IGS set-ups (mobile AR and desktop AR) and traditional navigation on attention shifts for the specific task of craniotomy planning. We found a significant difference in terms of the time taken to perform the task and attention shifts between traditional navigation, but no significant difference between the different AR set-ups. With mobile AR, however, users felt that the system was easier to use and that their performance was better. These results suggest that regardless of where the AR visualisation is shown to the surgeon, AR may reduce attention shifts, leading to more streamlined and focused procedures.

Keywords: augmented reality; augmented reality image-guided neurosurgery; craniotomy planning; desktop augmented reality; medical image processing; mobile augmented reality; neurophysiology; surgery; tumour; tumours.

Fig. 1

Specific task of outlining the extent of a tumour on the skull a A surgeon uses a pointer in his right hand to locate the boundary of the tumour and draws dots with his left hand at different locations b AR visualisation would allow the surgeon to see the tumour merged with the real surgical scene and can use that to draw the extent of the tumour. (b) Inlay: with traditional neuronavigation this surgeon has drawn dots using guidance and then connects the dots to create the contour of the tumour

Fig. 2

Screenshot of the IGNS monitor view for the traditional nav condition: the user has access to the pointer position (coloured cross-hair) as well as patient's preoperative scan and the segmented tumour model (green)

Fig. 3

Top: experimental set-up: the user holds the pointer in one hand and the marker in the other. Depending on the condition he or she looks either at the mobile phone (outfitted with a tracker) for the AR visualisation or on the desktop for either AR or IGNS navigation. The purpose of the task is to draw the contour of the tumour on the surface of the phantom. Bottom left: subjects’ point of view of the experimental set-up when testing the mobile AR condition. Bottom right: screenshot of the desktop AR view

Fig. 4

Boxplots of the total times taken per condition (in seconds). The average times to delineate a tumour were $50.78\pm 24.34$, $25.5\pm 10.95$ and $20.6\pm 8.23$ for traditional nav, desktop AR and mobile AR, respectively

Fig. 5

Boxplots of the number of attention shifts per condition. The average number of attention shifts were $27.8\pm 14.00$, $6.3\pm 7.79$ and $2.3\pm 2.93$ for traditional nav, desktop AR and mobile AR, respectively

Fig. 6

Boxplots of the ratio of time looking at desktop/mobile over total time per condition. The averages were $0.60\pm 0.18$, $0.91\pm 0.07$ and $0.95\pm 0.05$ for traditional nav, desktop AR and mobile AR, respectively

Fig. 7

Individual scales of the NASA–TLX for the different conditions, ranging from 0 to 10. The results show that for all measures mobile AR was perceived to be better/easier to use than desktop AR, which in turn performs better than traditional nav