High-Performance Virtual Reality Volume Rendering of Original Optical Coherence Tomography Point-Cloud Data Enhanced With Real-Time Ray Casting

Abstract

Purpose: Feasibility testing of a novel volume renders technology to display optical coherence tomography data (OCT) in a virtual reality (VR) environment.

Methods: A VR program was written in C++/OpenGL to import and display volumetric OCT data in real time with 180 frames per second using a high-end computer and a tethered head-mounted display. Following exposure, participants completed a Simulator Sickness Questionnaire (SSQ) to assess for nausea, disorientation, and oculomotor disturbances. A user evaluation study of this software was conducted to explore the potential utility of this application.

Results: Fifty-seven subjects completed the user testing (34 males and 23 females). Mean age was 48.5 years (range, 21-77 years). Mean acquired work experience of the 35 ophthalmologists (61.40%) included in the group was 15.46 years (range, 1-37 years). Twenty-nine participants were VR-naïve. The SSQ showed a mean total score of 5.8 (SD = 9.44) indicating that the system was well tolerated and produced minimal side effects. No difference was reported between VR-naïve participants and experienced users. Overall, immersed subjects reported an enjoyable VR-OCT presence effect.

Conclusions: A usable and satisfying VR imaging technique was developed to display and interact with original OCT data.

Translational relevance: An advanced high-end VR image display method was successfully developed to provide new views and interactions in an ultra high-speed projected digital scenery using point-cloud OCT data. This represents the next generation of OCT image display technology and a new tool for patient engagement, medical education, professional training, and telecommunications.

Keywords: mesh; ophthalmology; optical coherence tomography; point cloud data; polygon; ray casting; virtual reality; volume rendering.

Figure 1

Stereoscopic illustration of the VR environment displaying volume OCT data of a peripheral retinal tear used in this study (images can be fused for a 3D sensation). (A) High-quality point-cloud rendering shows how the round border of the retinal tear is held with the left-hand VR handle (single arrow) and the origin of a retinal bridging vessel (white arrow head) is indicated with the right VR handle (double arrows). The neurosensory layer of the retina shows vitreoretinal traction (arrow head) and is separated from the underlying retinal pigment epithelium (RPE; double white arrow head), which is enhanced using shadow ray casting to provide realism and a sense of depth. Note that the used ray casting depicts near reality impression of floor scratches (asterisk) to produce the sensation of “being there” allowing reaction to stimuli as if they were in the real world, although the user is immersed in a synthetic environment. (B) The vitreoretinal traction is peeled off using a separate cutting plane (arrow) to highlight the bridging vessel (white arrow head). The cutting plane can be manipulated in all directions to deliver full freedom to operate. (C) Switch from VR-3D rendering (A, B) to original cross-section OCT mode in the same VR model and VR room shows the vitreoretinal traction (arrowhead) being continuous with the detached retina, which is separated from underlying RPE (double arrow head).

Figure 2

Visual representation of the percentage agreement levels in response to 12 statements probing the utility of the virtual reality (VR) tool. Overall, the full volume rendering tool is well tolerated and beneficial.

Figure 3

Integration of different image data resources into the VR-imaging method to show the capability of VR as a novel multimodal imaging display platform. (A) The developed VR tool was capable to import original, structural swept-source OCT data, for example of vitreoretinal traction and showing the corresponding original point-cloud rendering (OCT optical specimen measuring 9 × 12 mm; Topcon) (B). (C) Another example is shown as VR OCT angiography (3 × 3 mm, spectral-domain Cirrus HD-OCT; Carl Zeiss Meditec) and en face rendering of the optic disc vessels (D). (E) VR CT of a skull with soft tissue rendering and corresponding original CT data with intensity display (F) to show that VR as a new medium may be beneficial for ophthalmology and potentially for medical education and other healthcare subspecialties, such as neuroradiology or neurosurgery.