Development of a Mixed Reality Platform for Lateral Skull Base Anatomy

Abstract

Objectives: A mixed reality (MR) headset that enables three-dimensional (3D) visualization of interactive holograms anchored to specific points in physical space was developed for use with lateral skull base anatomy. The objectives of this study are to: 1) develop an augmented reality platform using the headset for visualization of temporal bone structures, and 2) measure the accuracy of the platform as an image guidance system.

Methods: A combination of semiautomatic and manual segmentation was used to generate 3D reconstructions of soft tissue and bony anatomy of cadaver heads and temporal bones from 2D computed tomography images. A Mixed-Reality platform was developed using C# programming to generate interactive 3D holograms that could be displayed in the HoloLens headset. Accuracy of visual surface registration was determined by target registration error between seven predefined points on a 3D holographic skull and 3D printed model.

Results: Interactive 3D holograms of soft tissue, bony anatomy, and internal ear structures of cadaveric models were generated and visualized in the MR headset. Software user interface was developed to allow for user control of the virtual images through gaze, voice, and gesture commands. Visual surface point matching registration was used to align and anchor holograms to physical objects. The average target registration error of our system was 5.76 mm ± 0.54.

Conclusion: In this article, we demonstrate that an MR headset can be applied to display interactive 3D anatomic structures of the temporal bone that can be overlaid on physical models. This technology has the potential to be used as an image guidance tool during anatomic dissection and lateral skull base surgery.

Figure 1.

Work flow demonstrating generation of 3D interactive hologram and projection onto cadaveric specimen as seen through the Microsoft HoloLens.

Figure 2.

Demonstration of measurement of target localization error (TLE). Panel A) Observer using 4-point passive probe (blue arrow) to measure TLE of selected landmarks with Optitrack camera bar (red arrow); Panel B) View through the HoloLens during measurement.

Figure 3.

3D interactive holograms superimposed on cadaveric temporal bone as seen through the HoloLens. Panel A) Skin and bone; Panel B) Bone only; Panel C) Internal Ear structures including jugular bulb (blue), carotid (red), facial nerve (yellow).