The deformable most-likely-point paradigm

Abstract

In this paper, we present three deformable registration algorithms designed within a paradigm that uses 3D statistical shape models to accomplish two tasks simultaneously: 1) register point features from previously unseen data to a statistically derived shape (e.g., mean shape), and 2) deform the statistically derived shape to estimate the shape represented by the point features. This paradigm, called the deformable most-likely-point paradigm, is motivated by the idea that generative shape models built from available data can be used to estimate previously unseen data. We developed three deformable registration algorithms within this paradigm using statistical shape models built from reliably segmented objects with correspondences. Results from several experiments show that our algorithms produce accurate registrations and reconstructions in a variety of applications with errors up to CT resolution on medical datasets. Our code is available at https://github.com/AyushiSinha/cisstICP.

Keywords: Deformable most-likely-point paradigm; Deformable registration; Shape inference; Statistical shape models.

Fig. A.15

Sample size experiment: translation (top) and rotation (bottom) errors produced using (from L to R) 1000 and 2000 sample points from the pelvis model in Exp. 1. Bars in the histogram are transparent to show all three algorithms.

Fig. A.16

Sample size experiment: translation (top) and rotation (bottom) errors produced using (from L to R) 1000 and 2000 sample points from the pelvis model in Exp. 2. Bars in the histogram are transparent to show all three algorithms.

Fig. A.17

Sample size experiment: mean tRE with increasing number of samples from Exp. 2 (shown with 500 and 2000 sample points for simplicity).

Fig. A.18

Sample size experiment: evolution of the quality of matched points for each algorithm with increasing iterations in Exp. 2 with 2000 sample points. The plot shows distances between matched points at each iteration and the location of points sampled from the deformed model shape. Added orientations drastically improve the quality of matched points.

Fig. A.19

Residual errors compared against tRE using 2000 sample points in Exp. 2 of the sample size experiment. The two measures exhibit correlation with correlation coefficients (left to right) of 0.85, 0.72 and 0.86. Results from trials using 0 modes are ignored here to focus on the deformable algorithms.

Fig. A.20

Noise model experiment: mean tREs produced by our algorithms (L-R) D-IMLP, D-IMLOP and GD-IMLOP in Exps. 1 (top), 2 (middle) and 3 (bottom). Note that the errors are increasing with increasing modes only because for this experiment the number of modes used to estimate the shapes equals the number of modes used to simulate a new shape from which points were sampled.

Fig. A.21

Residual errors compared against tRE using 500 sample points with 2 × 2 × 2 mm³ SD positional noise and 2° SD angular noise in Exp. 1 of the noise model experiment. The two measures exhibit correlation with correlation coefficients (L-R) of 0.86, 0.88 and 0.83.

Fig. A.22

Outlier experiment: mean tRE with different number of outliers for D-IMLP (top-left), D-IMLOP (top-right), and GD-IMLOP (bottom-left), and for all three algorithms using sample points with 0% outliers (bottom-right). Note that the errors are increasing with increasing modes only because for this experiment the number of modes used to estimate the shapes equals the number of modes used to simulate the deformed shape from which points were sampled.

Fig. A.23

Noise model experiment: mean tREs produced by our algorithms with different isotropic (left) and anisotropic (right) position noise assumptions, labeled on the x-axis, and different orientation noise assumptions, with standard deviations of 2° (top), 10° (middle) and 20° (bottom) in Exp. 4.

Fig. A.24

Non-medical data experiment: mean tRE (left) and tSE (right) obtained using different number of modes to estimate the test shape using facial expression data.

Fig. A.25

Scale experiment: Mean tSE (left) and tRE (middle) using sample points with 0% outliers and scale applied to the sampled points, and mean errors in recovering scale with increasing number of modes (right). Again, the number of modes used to estimate the shapes equals the number of modes used to simulate the deformed shape from which points were sampled.

Fig. A.26

Non-medical data experiment: this particular target shape (right) has a lot of detail which is necessary to convey the emotion in this face. 1000 sample points are too few to capture this detail resulting in an inaccurate reconstruction (left). However, with 2000 sample points, we are able to estimate this expression better (middle) since more sample points are better able to capture the detail in the target.

Fig. A.27

Non-medical data exp.: residual errors compared against tRE for GD-IMLOP using facial expression data. The two measures show correlation (correlation coefficient = 0.77).

Fig. 1

ICP iterates between finding point correspondences between data points, x_i, and model shape points, y_i, and computing the transformation that best aligns the matches.

Fig. 2

Inputs for D-IMLP: Mean mesh with modes (left), and point samples with positional noise model (right).

Fig. 3

Inputs for D-IMLOP (left) and GD-IMLOP (right): Mean mesh with normals and modes, and point samples with positional and isotropic (left) or anisotropic (right) orientation noise models.

Fig. 4

Registration metrics: tSE (left) measures the Hausdorff distance between the ground truth shape (green) and the shape estimated by our algorithm in shape space (blue), not taking the final transformation computed by the algorithm into consideration. tRE (right) measures the Hausdorff distance between the ground truth shape (green) and the estimated shape (blue) transformed to sample point space, therefore also adding the transformation computed by our algorithms into the error metric.

Fig. 5

An example of data generated for the leave-one-out experiment: points are sampled uniformly from the middle turbinate (left) and right nasal cavity (right) meshes.

Fig. 6

Leave-one-out experiment: mean tRE (top-left), tSE (top-right), translation and rotation errors (bottom) obtained using different number of modes to estimate the left-out middle turbinates and recover the transformations in Exp. 1.

Fig. 7

Leave-one-out experiment: mean time taken by our algorithms to compute registrations using different number of modes.

Fig. 8

Leave-one-out experiment: residual errors compared against tRE produced by D-IMLP (top-left), D-IMLOP (top-right), and GD-IMLOP (bottom-left), and error produced by CPD compared against the tSE (bottom-right) in Exp. 1. The two measures show correlation using D-IMLP, D-IMLOP and GD-IMLOP with coefficients 0.91, 0.65 and 0.61, respectively, but not using CPD (correlation coefficient = 0.05).

Fig. 9

Leave-one-out experiment: mean tRE (top-left), tSE (top-right), translation and rotation errors (bottom) obtained using different number of modes to estimate the left-out right nasal cavity meshes and recover the transformations in Exp. 2.

Fig. 10

An example of data generated for the partial data experiment: (left) points are sampled only from the ilium and ischium on the pelvis mesh, and (right) points are sampled from the front section of the right nostril which include parts of the septum and middle and inferior turbinates.

Fig. 11

Partial data experiment: mean tRE (top-left), tSE (top-right), translation and rotation errors (bottom) obtained using different number of modes to estimate the left-out pelvis meshes and recover the transformations in Exp. 1.

Fig. 12

Partial data experiment: mean tRE (top-left), tSE (top-right), translation and rotation errors (bottom) obtained using different number of modes to estimate the left-out right nasal cavity meshes and recover the transformation in Exp. 2.

Fig. 13

Partial data experiment: residual errors compared against tRE for GD-IMLOP in (L) Exp. 1 using pelvis data (correlation coefficient = 0.56) and (R) Exp. 2 using right nasal cavity data (correlation coefficient = 0.64).

Fig. 14

Clinical data experiment: With the first point set (top), registration results using D-IMLP (left) and D-IMLOP (middle) show failed registrations, while that using GD-IMLOP (right) shows good alignment (along with some outliers). The second point set (bottom) yields better results, with all three algorithms producing good alignments. However, we can see that the number of outliers or bad matches (red points matched to the outside of the nose) goes down as we go from D-IMLP (left) to D-IMLOP (middle) to GD-IMLOP (right).