Adaptive Physics-Based Non-Rigid Registration for Immersive Image-Guided Neuronavigation Systems

Abstract

Objective: In image-guided neurosurgery, co-registered preoperative anatomical, functional, and diffusion tensor imaging can be used to facilitate a safe resection of brain tumors in eloquent areas of the brain. However, the brain deforms during surgery, particularly in the presence of tumor resection. Non-Rigid Registration (NRR) of the preoperative image data can be used to create a registered image that captures the deformation in the intraoperative image while maintaining the quality of the preoperative image. Using clinical data, this paper reports the results of a comparison of the accuracy and performance among several non-rigid registration methods for handling brain deformation. A new adaptive method that automatically removes mesh elements in the area of the resected tumor, thereby handling deformation in the presence of resection is presented. To improve the user experience, we also present a new way of using mixed reality with ultrasound, MRI, and CT. Materials and methods: This study focuses on 30 glioma surgeries performed at two different hospitals, many of which involved the resection of significant tumor volumes. An Adaptive Physics-Based Non-Rigid Registration method (A-PBNRR) registers preoperative and intraoperative MRI for each patient. The results are compared with three other readily available registration methods: a rigid registration implemented in 3D Slicer v4.4.0; a B-Spline non-rigid registration implemented in 3D Slicer v4.4.0; and PBNRR implemented in ITKv4.7.0, upon which A-PBNRR was based. Three measures were employed to facilitate a comprehensive evaluation of the registration accuracy: (i) visual assessment, (ii) a Hausdorff Distance-based metric, and (iii) a landmark-based approach using anatomical points identified by a neurosurgeon. Results: The A-PBNRR using multi-tissue mesh adaptation improved the accuracy of deformable registration by more than five times compared to rigid and traditional physics based non-rigid registration, and four times compared to B-Spline interpolation methods which are part of ITK and 3D Slicer. Performance analysis showed that A-PBNRR could be applied, on average, in <2 min, achieving desirable speed for use in a clinical setting. Conclusions: The A-PBNRR method performed significantly better than other readily available registration methods at modeling deformation in the presence of resection. Both the registration accuracy and performance proved sufficient to be of clinical value in the operating room. A-PBNRR, coupled with the mixed reality system, presents a powerful and affordable solution compared to current neuronavigation systems.

Keywords: deep learning; deformable registration; machine learning; medical image computing; mesh generation; mixed reality; neuronavigation systems; neurosurgery.

Figure 1

Discrepancies between preoperative and intraoperative MR Imaging before and during neurosurgery. Left: preoperative MRI; Right: intraoperative MRI acquired after a part of the tumor is removed. The yellow outline indicates the preoperative brain outline after a rigid rotation. The large dark cavity is the tumor resection.

Figure 2

Overview of the OST IGNS framework. The left part is similar to traditional IGNS with the inclusion of a data server. The right part is the OST portion which includes a data receiver, a spatial registration method for conversion of objects to HoloLens space, and HoloLens rendering. The data server and data receiver communicate wirelessly.

Figure 3

A multi-tissue (brain parenchyma, tumor) finite element mesh used for non-rigid registration (number of tetrahedral elements: 160,179; minimum dihedral angle: 4.41°). Top row: the mesh superimposed on a volume rendering of the MRI data. Cyan and red represent the brain parenchyma and tumor meshes, respectively. Bottom row: mesh fidelity illustrated on an axial, sagittal, and coronal slices. Each slice depicts a 2D cross-section of the mesh surface (cyan and red lines) and the segmented volume (green and yellow regions). The closer the mesh surface is to the segmented boundaries, the higher the mesh fidelity.

Figure 4

Selected blocks from an MRI volume using various connectivity patterns. Blocks are depicted on 10 consecutive sagittal slices. From top to bottom row: sagittal slice (left) and volumetric MRI rendering (right); selected blocks with a “vertex” pattern; selected blocks with an “edge” pattern; selected blocks with a “face” pattern. Number of selected blocks for all patterns: 322,060.

Figure 5

Visualization of the metric construction for mesh adaptation. Left isotropic metric that set the spacing equal to the distance of the 5th closest registration point. Right anisotropic metric based on the five registration points for different values of the inflation parameter a.

Figure 6

Qualitative results. Each row represents the same slice of a 3D volume for the case numbered on the left. From left to right: intraoperative MRI (A); deformed preop MRI after (B) rigid registration, (C) B-Spline, (D) PBNRR, and (E) A-PBNRR; difference between intraoperative MRI and (F) B-Spline, (G) PBNRR, and (H): A-PBNRR.

Figure 7

Extracted Canny points in a single slice for quantitative evaluation of registration accuracy using the HD metric. (A): Points extracted from the preoperative MRI; (B): Points of (A) after transformation to the intraoperative space; (C): Points extracted from the intraoperative MRI. The HD metric is computed between point sets (B) and (C). Note that the Canny points are generally different from feature points used for registration.

Figure 8

(A) Plot of Hausdorff Distance (HD) errors of Table 4. Brain shift: cases 1–7; partial resection: cases 8–12; total resection: cases 13–25; supra total resection: cases 26–30. (B) End-to-end execution times for registration from preoperative to intraoperative images (excluding B-Spline times of more than about 8 min). All registration methods were run in parallel on 12 hardware cores on a DELL workstation with 12 Intel Xeon X5690@3.47 GHz CPU cores and 96 GB of RAM. Execution times include I/O. The mesh generation time is excluded from the PBNRR (preoperative step) but is included in the A-PBNRR (intraoperative step).

Figure 9

Anatomical landmarks (A–F) used for quantitative evaluation of registration accuracy. A neurosurgeon located the landmarks. (A,B) cortex near tumor; (C) anterior horn of later ventricle; (D) triangular part of lateral ventricle; (E) junction between pons and mid-brain; (F) roof of fourth ventricle.