Augmented Endoscopic Images Overlaying Shape Changes in Bone Cutting Procedures

Abstract

In microendoscopic discectomy for spinal disorders, bone cutting procedures are performed in tight spaces while observing a small portion of the target structures. Although optical tracking systems are able to measure the tip of the surgical tool during surgery, the poor shape information available during surgery makes accurate cutting difficult, even if preoperative computed tomography and magnetic resonance images are used for reference. Shape estimation and visualization of the target structures are essential for accurate cutting. However, time-varying shape changes during cutting procedures are still challenging issues for intraoperative navigation. This paper introduces a concept of endoscopic image augmentation that overlays shape changes to support bone cutting procedures. This framework handles the history of the location of the measured drill tip as a volume label and visualizes the remains to be cut overlaid on the endoscopic image in real time. A cutting experiment was performed with volunteers, and the feasibility of this concept was examined using a clinical navigation system. The efficacy of the cutting aid was evaluated with respect to the shape similarity, total moved distance of a cutting tool, and required cutting time. The results of the experiments showed that cutting performance was significantly improved by the proposed framework.

Fig 1. Microendoscopic discectomy for spinal disorders.

(a) Tool operation during surgery. (b) Endoscopic view with a 16-mm diameter. (c) Preoperative and (d) postoperative CT images. The yellow and black arrows show the cutting results.

Fig 2. Flowchart of augmented endoscopic image generation.

The remains to be cut for the target shape are visualized as augmented images based on the history of the drill tip during cutting.

Fig 3. Volume label definition.

(a) History of cutting label. The tip of the surgical tool is represented by a sphere, and the current cutting state is stored by the binary voxel values. (b) Definition of remains to be cut in the depth direction. The differential map is used for generating the AR image.

Fig 4. Comparison of color overlay methods.

Hue shift in the hue saturation value color space and opacity control applied to the microendoscopic images.

Fig 5. Hardware setup and workspace for experimental system.

(a) An electric router mounted on the PHANToM was used for tool tip measurement. (b) The workspace was covered and the subjects could only see camera images displayed in the monitor.

Fig 6. Augmented camera images as the cutting operation progresses.

The color represents the amount of cutting remaining for the target shape.

Fig 7. Augmented images for cutting aid.

(a) Local color shift and alpha blending with (b) 10% and (b) 20% opacity values. The local color shift achieves better visibility of textures and shapes in different illumination conditions.

Fig 8. AE images of a 3D printer model made from CT images with different camera orientations.

The direction and depth that should be cut on the spinal structure are visualized using the heat map.

Fig 9. VR images of four target shapes for cutting experiments.

(a) Semi-ellipsoid, (b) hemi-spherical, (c) cuboid, and (d) columnar.

Fig 10. Evaluation results of cutting aid using the proposed AR images and VR images for comparison.

(a) Cutting time E_t, (b) total distance of the electric router tip E_d, and (c) shape similarity of the target region E_s.

Fig 11. Virtual image generation using StealthStation.

(a) Metal tube with 16-mm diameter, (b) microendoscope attached to metal tube, (c) microendoscopic image, and (d) calibrated virtual image reflecting microscopic lens properties.

Fig 12. Registration accuracy evaluation using clinical navigation system.

(a) Evaluation points marked on the top of a wooden block and (b) reference probe attached with reflective markers.