A Virtual Reality System for Improved Image-Based Planning of Complex Cardiac Procedures

Abstract

The intricate nature of congenital heart disease requires understanding of the complex, patient-specific three-dimensional dynamic anatomy of the heart, from imaging data such as three-dimensional echocardiography for successful outcomes from surgical and interventional procedures. Conventional clinical systems use flat screens, and therefore, display remains two-dimensional, which undermines the full understanding of the three-dimensional dynamic data. Additionally, the control of three-dimensional visualisation with two-dimensional tools is often difficult, so used only by imaging specialists. In this paper, we describe a virtual reality system for immersive surgery planning using dynamic three-dimensional echocardiography, which enables fast prototyping for visualisation such as volume rendering, multiplanar reformatting, flow visualisation and advanced interaction such as three-dimensional cropping, windowing, measurement, haptic feedback, automatic image orientation and multiuser interactions. The available features were evaluated by imaging and nonimaging clinicians, showing that the virtual reality system can help improve the understanding and communication of three-dimensional echocardiography imaging and potentially benefit congenital heart disease treatment.

Keywords: echocardiography; pre-operative imaging; virtual reality.

Figure 1

System overview (left) including the hardware (VR set and computer) and software (Unity application connected to Python and to VTK through Unity’s native plug-ins) and an example of a screen capture of our virtual reality application (right). This case shows a cut of a 3D ultrasound image of the left ventricle with three highlighted landmarks (TV—tricuspid valve, AoV—aortic valve, MV—mitral valve) and a multiplanar reconstruction plane showing the cropping slice.

Figure 2

Illustration of the different interactions featured in our VR system, controlled using the blue cross-hair widget attached to the controllers. (a) Rotations and translations are highlighted using the volume bounding box. Scaling is applied by pulling the corners of the bounding box. (b) Cropping uses a transparent red plane that cuts into the volume. (c) Landmarks include a pin point, a label line and a label and can be moved and stretched with the cross-hair tool. (d) The linear measurement tool allows the user to draw a green dashed line between two end points and displays the length in mm.

Figure 3

Multiuser features. (a) Representation of synchronous collaborative planning, where two users simultaneously interrogate the data remotely. Each user sees the other users by their virtual avatar. (b) Interface for asynchronous collaboration, where one user can play back an interrogation session previously carried out by another user. The playback is a single-player feature who can toggle the display of the avatar who performed the recorded interaction.

Figure 4

Experimental features investigated using our VR system. (a) Visualisation of a CFD blood flow simulation in the right ventricle of an HLHS patient, showing blood velocity with streamlines and a user-defined plane with the pressure distribution over that plane. (b) User opinion on using haptic feedback during a measurement task, showing overall preference for enabled haptics. (c) Automatic anatomical orientation of the rendered volume that is aligned with a cardiac model.

Figure 5

Comparison of image quality between QLAB and the VR system. (a) QLAB image with colour depth cueing, where closer tissues are brown and farther tissue are blue. (b) VR image with a similar colour map, but without colour depth cueing.

Figure 6

Bland–Altman plots comparing line measurements made using our VR system and Tomtec CardioView to Philips QLAB. Our VR system’s measurements show less bias than Tomtec, but greater variability.