Multimodal Registration for Image-Guided EBUS Bronchoscopy

Abstract

The state-of-the-art procedure for examining the lymph nodes in a lung cancer patient involves using an endobronchial ultrasound (EBUS) bronchoscope. The EBUS bronchoscope integrates two modalities into one device: (1) videobronchoscopy, which gives video images of the airway walls; and (2) convex-probe EBUS, which gives 2D fan-shaped views of extraluminal structures situated outside the airways. During the procedure, the physician first employs videobronchoscopy to navigate the device through the airways. Next, upon reaching a given node's approximate vicinity, the physician probes the airway walls using EBUS to localize the node. Due to the fact that lymph nodes lie beyond the airways, EBUS is essential for confirming a node's location. Unfortunately, it is well-documented that EBUS is difficult to use. In addition, while new image-guided bronchoscopy systems provide effective guidance for videobronchoscopic navigation, they offer no assistance for guiding EBUS localization. We propose a method for registering a patient's chest CT scan to live surgical EBUS views, thereby facilitating accurate image-guided EBUS bronchoscopy. The method entails an optimization process that registers CT-based virtual EBUS views to live EBUS probe views. Results using lung cancer patient data show that the method correctly registered 28/28 (100%) lymph nodes scanned by EBUS, with a mean registration time of 3.4 s. In addition, the mean position and direction errors of registered sites were 2.2 mm and 11.8∘, respectively. In addition, sensitivity studies show the method's robustness to parameter variations. Lastly, we demonstrate the method's use in an image-guided system designed for guiding both phases of EBUS bronchoscopy.

Keywords: bronchoscopy; endobronchial ultrasound; image registration; image-guided surgery systems; lung cancer; multimodal imaging.

Figure 1

Device tip and example views for an integrated EBUS bronchoscope. The video camera provides bronchoscopic video, while the EBUS transducer gives 2D EBUS views. The magenta and blue arrows correspond to videobronchoscope camera axis ${\mathbf{n}}_{\mathrm{C}}$ and EBUS probe axis ${\mathbf{n}}_{\mathrm{US}}$, respectively.

Figure 2

Overview of an image-guided EBUS bronchoscopy procedure. Top part of the figure focuses on the 3D CT-based virtual chest space, whereas the bottom part features the analogous real 3D chest space. (a) Before the live procedure, the physician selects lymph nodes of interest from a patient’s chest CT scan. (b) During the live surgical procedure, the physician then uses this plan to navigate the bronchoscope toward the lymph node. (c) When using an image-guided bronchoscopy system, the physician receives additional graphical feedback on how to navigate the device toward the lymph node. VB views situated along a precomputed guidance path (blue line) lead the physician toward the lymph node. Image registration between the CT-based VB views and live bronchoscopic video (red dotted-line box) facilitates device synchronization during navigation and leads the physician to the proper airway closest to the node (green region). (d) Existing image-guided bronchoscopy systems offer no means for helping to place the EBUS probe for live localization of the extraluminal lymph node. (Bronchoscopy drawing by Terese Winslow, “Bronchoscopy,” NCI Visuals Online, National Cancer Institute.)

Figure 3

Example videobronchoscope and EBUS views constituting the multimodal EBUS bronchoscope. (Top) bronchoscopic video sources for the real EBUS bronchoscope and virtual device. (Bottom) corresponding fan-shaped EBUS views, ${\mathbf{I}}_{\mathrm{US}}$ and ${\mathbf{I}}_{\mathrm{CT}}$, for the real and virtual devices, respectively. For virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}$, the blue lines demarcate the EBUS FOV, the magenta line indicates the video camera’s viewing direction ${\mathbf{n}}_C$, and the green region denotes a lymph node predefined in the chest CT scan. In this figure, the videobronchoscope and EBUS view pairs are not at the same site. In addition, both view pairs are registered.

Figure 4

EBUS probe model: (a) device tip model; (b) 2D EBUS probe view; red lines denote the 60${}^{\circ }$ fan-shaped view. Standard EBUS display settings were used throughout (gain = 19; contrast = 6).

Figure 5

CT-EBUS registration for a station-10 lymph node (case 21405-139). (a) Initial raw EBUS view ${\mathbf{I}}_{\mathrm{US}}$ (left) and corresponding CT-based virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}^{p_i}$ (right). (b) Result after segmenting the nodal ROI ${\mathbf{R}}_{\mathrm{US}}$ in EBUS view ${\mathbf{I}}_{\mathrm{US}}$; the green outline signifies the ROI contour [42]. (c) Registered pair (${\mathbf{I}}_{\mathrm{US}}$, ${\mathbf{I}}_{\mathrm{CT}}^{p_o}$) after final registration; ${\mathbf{I}}_{\mathrm{US}}$ depicts the fused registered CT-based ROI. In all virtual EBUS views, green regions denote nodal ROIs ${\mathbf{R}}_{\mathrm{CT}}$ predefined during planning, whereas red regions represent blood vessels.

Figure 6

Locating airway wall point $p_s$. Green dot denotes the current surface point $p_i$, yellow dot denotes the candidate position $p_v$, red dot is the closest k-d tree point $p_s$ to $p_v$, cyan dot is the closest airway centerline point $p_l$ to $p_v$, and orange hollow dot is the correct surface voxel $p_s^{\prime }$.

Figure 7

Limiting region of calculations for (5) in cost ${\mathbf{C}}_{\mathbf{N}}$. (a) Trapezoidal EBUS region delineated by the lines encompassing the EBUS ROI in ${\mathbf{I}}_{\mathrm{US}}$ with (b) showing segmented ROI ${\mathbf{R}}_{\mathrm{US}}$. (c,d) Corresponding virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}^p$ and predefined ROI ${\mathbf{R}}_{\mathrm{CT}}$.

Figure 8

CT-EBUS registration examples. (a–c) Station 4L node for case 20349-3-84. (d–f) Station 4L node for case 21405-108. Parts (a,d) show the automatically segmented ROI in EBUS frame ${\mathbf{I}}_{\mathrm{US}}$ using [42]. Parts (b,e) show the registered CT-based predefined ROI superimposed on ${\mathbf{I}}_{\mathrm{US}}$. Parts (c,f) depict the CT-based virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}^{p_o}$ after registration. In all views, the green region corresponds to the lymph node, whereas the red regions represent major vessels (PA = pulmonary artery).

Figure 9

Image-guided EBUS bronchoscopy for a station 4R lymph node for patient 21405-116. (a) 3D airway tree rendering and whole-body PET projection image indicating the target lymph node (red). (b) Registered real video and VB view after navigation. (c) Registered real EBUS view ${\mathbf{I}}_{\mathrm{US}}$, virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}$ (green region = node), and a CT-based simulated EBUS view, respectively, at final site.

Figure 10

Image-guided EBUS bronchoscopy for a station 4R lymph node for patient 20349-3-87. (a) 3D airway tree rendering and coronal fused CT/PET section indicating the target lymph node (color scale bar indicate PET SUV value). (b) Registered real EBUS view ${\mathbf{I}}_{\mathrm{US}}$, virtual EBUS view ${\mathbf{I}}_{\mathrm{CT}}$ (green region = node), and a CT-based simulated EBUS view, respectively, at final site.