Application of tolerance limits to the characterization of image registration performance

Abstract

Deformable image registration is used increasingly in image-guided interventions and other applications. However, validation and characterization of registration performance remain areas that require further study. We propose an analysis methodology for deriving tolerance limits on the initial conditions for deformable registration that reliably lead to a successful registration. This approach results in a concise summary of the probability of registration failure, while accounting for the variability in the test data. The (β, γ) tolerance limit can be interpreted as a value of the input parameter that leads to successful registration outcome in at least 100β% of cases with the 100γ% confidence. The utility of the methodology is illustrated by summarizing the performance of a deformable registration algorithm evaluated in three different experimental setups of increasing complexity. Our examples are based on clinical data collected during MRI-guided prostate biopsy registered using publicly available deformable registration tool. The results indicate that the proposed methodology can be used to generate concise graphical summaries of the experiments, as well as a probabilistic estimate of the registration outcome for a future sample. Its use may facilitate improved objective assessment, comparison and retrospective stress-testing of deformable.

Fig. 1

Estimation of tolerance limit for the synthetic deformation example in Example 1. Top: Result of the model fitting for the registration failure rate as a function of the initial misalignment. Each line corresponds to the modeled registration failure rate for an individual pair of images. Bottom: Mean failure rate with the 95% prediction interval. The (0.90,0.95) LTL corresponds to the intersection of the vertical blue line with the upper prediction interval and is equal to 8.7 mm.

Fig. 2

Example of the registration results for one of the cases used in the Example 2. Top: Axial slice of the intra-procedural T2w MRI with the contour of the capsule (green outline), and two of the fiducial points used in the evaluation (white arrows). The first fiducial (on the left) corresponds to the center of gravity for the segmentation of the dark round area. The second fiducial is at the corner formed by the ejuculatory ducts and the urethra. Bottom: Registered image, arrows point to the locations of the landmarks in the fixed image, which are close to the anatomical locations corresponding to the landmarks in the registered image.

Fig. 3

Estimation of tolerance limit for the registration experiment from Example 3. Top: Result of modeling for the estimated probability of failure as functions of misalignment norm. Each curve corresponds to a sample (pair of images being registered). As a result of this modeling, both intra- (estimated by the locfit procedure) and inter-sample variability can be estimated, as needed for the calculation of tolerance limits. Bottom: Average probability of failed registration with 95% prediction limit (dotted line) and (0.90,0.95) lower tolerance limit 2.8 mm (blue cross hairs).

Fig. 4

Cumulative number of failures versus misalignment norm for two representative samples from Example 2. The step function in black indicates the data, the simple linear logistic model fit is in red, and the nonparametric logistic regression fit is in blue. Plots correspond to sample 1 (left) and 8 (right). Improvement in the quality of fit relative to the linear logistic model fit was concluded based on the visual analysis and ROC quantitative assessment.

Fig. 5

Modeling of the tolerance limit for the registration experiment with multiple input parameters being varied (Example 3). Mean rate of failure (solid line) with the 95% prediction interval (dashed line) and LTL corresponding to the intersection of the blue vertical line with the upper prediction interval line. Top: Results of modeling that take into account all 10 cases, LTL is undefined. Bottom: Modeling results after excluding the one case that exhibited very frequent failures. Inclusion of the case with frequent failures has dramatic effect on LTL, while the average failure rate remains largely unchanged (misalignment norm corresponding to the 10% average probability of success changes from 3 to 4 mm).