Advanced Medical Use of Three-Dimensional Imaging in Congenital Heart Disease: Augmented Reality, Mixed Reality, Virtual Reality, and Three-Dimensional Printing

Abstract

Three-dimensional (3D) imaging and image reconstruction play a prominent role in the diagnosis, treatment planning, and post-therapeutic monitoring of patients with congenital heart disease. More interactive and realistic medical experiences take advantage of advanced visualization techniques like augmented, mixed, and virtual reality. Further, 3D printing is now used in medicine. All these technologies improve the understanding of the complex morphologies of congenital heart disease. In this review article, we describe the technical advantages and disadvantages of various advanced visualization techniques and their medical applications in the field of congenital heart disease. In addition, unresolved issues and future perspectives of these evolving techniques are described.

Keywords: 3D imaging; 3D modeling; 3D printing; Augmented reality; Congenital heart disease; Virtual reality.

Fig. 1. 3D cardiac MRI.

A. Oblique coronal view of time-resolved non-ECG-synchronized contrast-enhanced 3D cardiac MRI showing motion artifacts (arrows) in AA and LV, which is problematic in illustrating accurate cardiovascular morphology on advanced visualization techniques. B. Oblique coronal view of ECG-triggered, navigator-gated T2-prepared balanced steady-state free precession imaging demonstrating motionless cardiovascular morphology but suboptimal heterogeneous signal intensities in LA, LV, and RPA. AA = ascending aorta, ECG = electrocardiography, LA = left atrium, LV = left ventricle, MRI = magnetic resonance imaging, RPA = right pulmonary artery, 3D = three-dimensional

Fig. 2. Advanced post-processing of 3D cardiothoracic CT imaging.

A. In coronal volume-rendered CT images with major aortopulmonary collateral arteries, each artery can be illustrated in distinct color. In addition, other structures such as airways and lungs (B) and thoracic cage (C) may be rendered simultaneously to enhance their spatial relationships with collateral arteries, which is helpful for pre-procedural planning. CT = computed tomography

Fig. 3. CT ventricular volumetry using 3D threshold-based segmentation.

Coronal volume-rendered CT images highlight LV (A) and RV (B) segmented for ventricular volumetry using 3D threshold-based approach. RV = right ventricle

Fig. 4. Transparent-lumen cinematic rendering.

Depth perception of cinematic-rendered images (B, D) are superior to those on regular volume-rendered images (A, C). Therefore, papillary muscles (arrows; A, B) and trabeculations are better visualized on cinematic-rendered images (B, D). With improved depth perception, anterior marginal muscular VSDs (arrows; C, D) are demonstrated without additional back clipping that must be used for regular volume-rendered image (C). Realistic endocardial appearance of transparent-lumen cinematic rendering mimics 3D printed, hollow cardiac model. RA = right atrium, VSD = ventricular septal defect

Fig. 5. Workflow of advanced visualization technology.

Segmented and refined 3D model can be used not only for augmented reality, virtual reality, and interactive web or mobile 3D displays but also for 3D printing. STL = Standard Tessellation Language

Fig. 6. 3D printing workflow tailored to congenital heart disease.

A. Axial CT image shows segmented blood pool mask and segmented myocardium mask with distinct colors. B. 3D virtual cardiac myocardial (upper) and blood pool (lower) models. C. 3D printed cardiac myocardial (upper) and blood pool (lower) models. Small VSD (large arrow) needs to be widened in direction of small arrows.

Fig. 7. Patient-specific 3D printed hollow cardiac model using flexible printing material (TangoPlus, Stratasys) from 3D cardiothoracic CT data obtained preoperatively in infant with double outlet RV and interrupted aortic arch type B.

Both smaller AA and dilated MPA arise from double outlet RV. Aortic arch is interrupted (asterisk) between left common carotid artery and left subclavian artery, indicating type B of interrupted aortic arch. MPA = main pulmonary artery

Fig. 8. Direct file conversion from segmented DICOM file to STL file.

Segmented cardiovascular volume in DICOM format can be directly converted to STL format. Segmented part may be single volume (A, B) or merged volume comprising multiple parts (C). DICOM = Digital Imaging and Communication in Medicine

Fig. 9. 3D printed aortic valve model for surgical simulation.

A. Aortic valve (yellow) is seen in exported polygon rendered STL file. B. 3D printed model is used for surgical simulation of aortic valve. Surgeon may open aortic valve with their finger.