Reconstructing a 3D heart surface with stereo-endoscope by learning eigen-shapes

Abstract

An efficient approach to dynamically reconstruct a region of interest (ROI) on a beating heart from stereo-endoscopic video is developed. A ROI is first pre-reconstructed with a decoupled high-rank thin plate spline model. Eigen-shapes are learned from the pre-reconstructed data by using principal component analysis (PCA) to build a low-rank statistical deformable model for reconstructing subsequent frames. The linear transferability of PCA is proved, which allows fast eigen-shape learning. A general dynamic reconstruction framework is developed that formulates ROI reconstruction as an optimization problem of model parameters, and an efficient second-order minimization algorithm is derived to iteratively solve it. The performance of the proposed method is finally validated on stereo-endoscopic videos recorded by da Vinci robots.

Fig. 1

Illustration of model-based ROI reconstruction

Fig. 2

Stereo-endoscopic videos (I-IV) for validation

Fig. 3

PCA of shape data on videos I-IV

Fig. 4

Mean shapes and 1st eigen-shapes calculated from all 800 frames vs. from the first 600 frames. For each video, the first row shows the mean shapes, the second row the 1st eigen-shapes; the left shapes are from all 800 frames, the right shapes from the first 600 frame.

Fig. 5

Reconstructed ROI with the statistical deformable models on various videos. (Visualization 1, Visualization 2, Visualization 3, Visualization 4)

Fig. 6

Reconstructed ROI with the 4-CP TPS on video I.

Fig. 7

Motion trajectory analysis of ROI (p_o) tracked by the statistical deformable model.