A low-cost multimodal head-mounted display system for neuroendoscopic surgery

Abstract

Background: With rapid advances in technology, wearable devices as head-mounted display (HMD) have been adopted for various uses in medical science, ranging from simply aiding in fitness to assisting surgery. We aimed to investigate the feasibility and practicability of a low-cost multimodal HMD system in neuroendoscopic surgery.

Methods: A multimodal HMD system, mainly consisted of a HMD with two built-in displays, an action camera, and a laptop computer displaying reconstructed medical images, was developed to assist neuroendoscopic surgery. With this intensively integrated system, the neurosurgeon could freely switch between endoscopic image, three-dimensional (3D) reconstructed virtual endoscopy images, and surrounding environment images. Using a leap motion controller, the neurosurgeon could adjust or rotate the 3D virtual endoscopic images at a distance to better understand the positional relation between lesions and normal tissues at will.

Results: A total of 21 consecutive patients with ventricular system diseases underwent neuroendoscopic surgery with the aid of this system. All operations were accomplished successfully, and no system-related complications occurred. The HMD was comfortable to wear and easy to operate. Screen resolution of the HMD was high enough for the neurosurgeon to operate carefully. With the system, the neurosurgeon might get a better comprehension on lesions by freely switching among images of different modalities. The system had a steep learning curve, which meant a quick increment of skill with it. Compared with commercially available surgical assistant instruments, this system was relatively low-cost.

Conclusions: The multimodal HMD system is feasible, practical, helpful, and relatively cost efficient in neuroendoscopic surgery.

Keywords: head‐mounted display; neuroendoscopic surgery; surgical assistant instruments; virtual endoscopy.

Figure 1

Schematic view of the overall configuration of the multimodal head‐mounted display system for neuroendoscopic surgery. DVI, digital video interactive; DICOM, digital imaging and communications in medicine

Figure 2

Patient's MRI image and reconstructed virtual endoscopy image of cerebral ventricles. (A and B) T1 and T2 images showed lateral ventricles, and the third ventricle had enlarged, and the surrounding brain tissue was oedematous; (C) 3D‐SPACE image showed that the cerebral aqueduct seemed obstructed; (D) 3D reconstruction of ventricular system with software 3D Slicer; (E) virtual endoscopy image demonstrated that the interventricular foramen (red line) was enlarged; (F) the third ventricle failed to connect to the fourth ventricle because the cerebral aqueduct was obstructed

Figure 3

Endoscopic third ventriculostomy with the help of head‐mounted multimodal display system. (A) Keyhole craniotomy under action camera view; (B) endoscope‐assisted observation of intraventricular structure; (C) observation and adjustment of patient's reconstructed 3D virtual endoscopy image; (D) creation of an opening in the floor of the third ventricle under endoscopic view

Figure 4

Learning curve of the multimodal head‐mounted display system. Quadratic regression analysis of the operative time and number of cases experienced