Enhanced volumetric visualization for real time 4D intraoperative ophthalmic swept-source OCT

Abstract

Current-generation software for rendering volumetric OCT data sets based on ray casting results in volume visualizations with indistinct tissue features and sub-optimal depth perception. Recent developments in hand-held and microscope-integrated intrasurgical OCT designed for real-time volumetric imaging motivate development of rendering algorithms which are both visually appealing and fast enough to support real time rendering, potentially from multiple viewpoints for stereoscopic visualization. We report on an enhanced, real time, integrated volumetric rendering pipeline which incorporates high performance volumetric median and Gaussian filtering, boundary and feature enhancement, depth encoding, and lighting into a ray casting volume rendering model. We demonstrate this improved model implemented on graphics processing unit (GPU) hardware for real-time volumetric rendering of OCT data during tissue phantom and live human surgical imaging. We show that this rendering produces enhanced 3D visualizations of pathology and intraoperative maneuvers compared to standard ray casting.

Keywords: (100.0100) Image processing; (110.4500) Optical coherence tomography; (170.4460) Ophthalmic optics and devices.

Fig. 1

Left: Flowchart for the enhanced ray casting pipeline. Steps in the conventional ray casting pipeline are shown in square blocks while additions are shown in oval blocks. Right: Diagram illustrating ray casting in two dimensions.

Fig. 2

Left: SS-MIOCT system in use during human macular surgery. Right: Resolution and fall off plot for the SS-MIOCT system.

Fig. 3

Computation of the median via forgetful selection for a sample region of 7 pixels [54].

Fig. 4

Progressive addition of steps in the enhanced volume rendering pipeline (see Visualization 1): (a) basic ray casting (b) volumetric filtering (c) edge enhancement (d) feature enhancement (e) depth based shading (f) Phong shading. The volume is of coated forceps lifting a blood vessel in ex-vivo porcine retina. Volumes were acquired at 300 A-scans per B-scan and 128 B-scans per volume.

Fig. 5

Comparison of regular ray casting (left), enhanced ray casting with retinal settings (center), and enhanced ray casting with anterior segment settings (right) of the anterior segment of a healthy of a consented subject. Complex conjugate resolved SS-OCT imaging was performed using a long depth sample arm. Volumes were acquired at 1000 A-scans per B-scan and 128 B-scans per volume.

Fig. 6

Left: Timing diagram showing the acquisition and rendering pipeline for multiple groups of 16 B-scans. Right: Expanded view of a single acquisition. Kernels in the acquisition, filtering, and rendering pipelines are indicated with blue, purple, and green boxes respectively

Fig. 7

Intraoperative MIOCT volume of forceps peeling a membrane rendered as a stereoscopic pair, with a 9 degree offset. This data was displayed live to the surgeon through a novel stereoscopic heads-up display incorporated into the surgical microscope. Volumes were acquired at 300 A-scans per B-scan and 128 B-scans per volume over a 5mmx5mm region. Volumes were re-rendered for display outside of the OR using the same pipeline as the intraoperative software.

Fig. 8

Comparison of intraoperative volumes with (left) and without (right) enhanced volume rendering (see Visualization 2). Each pair of renders were taken from the same angle and annotations were placed at the same location in the renders. Top: Intraoperative visualization of a pre vitrectomy macular hole (MH) with an epiretinal membrane (ERM) causing macular pucker with (left) and without (right) enhanced volume rendering. Middle: Visualization of the same volume from a different angle. Bottom: Image of the macula post vitrectomy and ILM peel showing the remainder of the ERM, the macular hole starting to close and the foveal elevations (FE) caused by the surgeon’s forceps. Volumes were taken at 512 A-scans per B-scan and 128 B-scans per volume over a 5mmx5mm region. Renders were re-rendered for display outside of the OR using the same pipeline as the intraoperative software.

Fig. 9

Volume visualization (top) and B-scans (bottom) of an inter-operative ILM peel around a macular hole (MH) with a diamond dusted membrane scraper (MS). The track (T) of the instrument across the retina left a transient groove in the surface which is highlighted in red. Volumes were acquired at 300 A-scans per B-scan and 128 B-scans per volume over a 5mmx5mm region. B-scans are tracked to the tool location in post processing. Renders were re-rendered for display outside of the OR using the same pipeline as the intraoperative software.

Fig. 10

Visualizations of pathologic neovascularization in retinopathy of prematurity in an infant who was born three months premature and was imaged two months later. Fan-shaped neovascularization (NV) is visible on the back left corner with smaller neovascular buds (NB) at the right. Volumes were acquired with a Bioptigen hand held SD-OCT system at 1000 A-scans per B-scan and 64 B-scans per volume.