Robust Motion Estimation and Structure Recovery from Endoscopic Image Sequences With an Adaptive Scale Kernel Consensus Estimator

Abstract

To correctly estimate the camera motion parameters and reconstruct the structure of the surrounding tissues from endoscopic image sequences, we need not only to deal with outliers (e.g., mismatches), which may involve more than 50% of the data, but also to accurately distinguish inliers (correct matches) from outliers. In this paper, we propose a new robust estimator, Adaptive Scale Kernel Consensus (ASKC), which can tolerate more than 50 percent outliers while automatically estimating the scale of inliers. With ASKC, we develop a reliable feature tracking algorithm. This, in turn, allows us to develop a complete system for estimating endoscopic camera motion and reconstructing anatomical structures from endoscopic image sequences. Preliminary experiments on endoscopic sinus imagery have achieved promising results.

Figure 1

Simultaneous scale estimate of inliers and outlier detection. (a). The detected peaks and valleys with incorrect model parameters (b) and correct model parameters (c).

Figure 2

The histogram of ASKC scores of 10000 random samples. By way of illustration,

Figure 3

The procedure of the ASKC estimator

Figure 4

Lines extracted by the robust estimators.

Figure 5

Planes extracted by the robust estimators.

Figure 6

(a) and (b) a pair of original sinus endoscopic images; (c) the matches obtained by the SVD-matching algorithm; (d) the matches selected by the ASKC estimator on the left undistorted image; (e) and (f) the recovered epipolar geometry.

Figure 7

Overview of the SIFT feature tracking algorithm.

Figure 8

The trajectories of the tracked SIFT features.

Figure 9

Overview of the reconstruction algorithm.

Figure 10

Top row: the first, middle and last frame of the image sequence. Middle row: the recovered structure corresponding to the top row. Blue points are the newly recovered 3D points. Bottom row: (left) another view of the final recovered structure; (middle and right) two views of the final recovered structure by M1.