Image2SSM: Reimagining Statistical Shape Models from Images with Radial Basis Functions

Abstract

Statistical shape modeling (SSM) is an essential tool for analyzing variations in anatomical morphology. In a typical SSM pipeline, 3D anatomical images, gone through segmentation and rigid registration, are represented using lower-dimensional shape features, on which statistical analysis can be performed. Various methods for constructing compact shape representations have been proposed, but they involve laborious and costly steps. We propose Image2SSM, a novel deep-learning-based approach for SSM that leverages image-segmentation pairs to learn a radial-basis-function (RBF)-based representation of shapes directly from images. This RBF-based shape representation offers a rich self-supervised signal for the network to estimate a continuous, yet compact representation of the underlying surface that can adapt to complex geometries in a data-driven manner. Image2SSM can characterize populations of biological structures of interest by constructing statistical landmark-based shape models of ensembles of anatomical shapes while requiring minimal parameter tuning and no user assistance. Once trained, Image2SSM can be used to infer low-dimensional shape representations from new unsegmented images, paving the way toward scalable approaches for SSM, especially when dealing with large cohorts. Experiments on synthetic and real datasets show the efficacy of the proposed method compared to the state-of-art correspondence-based method for SSM.

Keywords: Deep Learning; Polyharmonic Splines; Radial Basis Function Interpolation; Statistical Shape Modeling.

Fig. 1.

(a) Concept of populating a surface using control points and the iso-surfaces using positive and negative pole points. (b) Same concept applied to an output three-dimensional reconstructed femur. (c) Normals can be used to describe very distinct features of the greater trochanter.

Fig. 2.

The Image2SSM architecture. A 3D image is fed to the convolutional backbone, which produces a flattened output for the feature extractor to produce control points and their respective normals. These are then used to compute the losses of the network.

Fig. 3.

(a) First and second modes of variation obtained from Image2SSM training data and PSM. (b) Surface-to-surface distance on a best, median, and worst training femur mesh. (c) The left image shows the surface-to-surface distance comparison on all the data used to train Image2SSM; the right shows it without outliers.

Fig. 4.

(a) Surface-to-surface distance on a reconstructed femur mesh from particles of a few test samples. (b) Surface-to-surface distance plot between DeepSSM and Image2SSM, and the same plot without the outlier femur. (c) Illustrates Image2SSM’s capacity to capture detail on an unseen test image. (d) Shows the compactness (higher is better), specificity (lower is better) and generalization (lower is better) graphs against the number of modes of variation.

Fig. 5.

We also demonstrate our results on a dataset of 1018 aligned left atrium MRI image-segmentation pairs. This dataset is very challenging due to the high variability in the manual labeling of the pulmonary arteries and the presence of various atrial fibrillation phenotypes (Persistent, paroxysmal, AFL, nonAF, other arrhythmia) As before, we build the model with 128 particles. We show the first three modes of variation of Image2SSM compared to PSM. The results are comparable and match expectations. We observe that both models capture the shape variability of the atrium itself well, less so with the pulmonary arteries.

Fig. 6.

(a) Surface-to-surface distance on a best, median, and worst training meshes. (b) The surface-to-surface distance comparison on all the data used to train Image2SSM. We observe that the distances are comparable between both models in that they capture a large array of shapes well, but fail to different degrees on severe outliers. (c) Shows the compactness (higher is better), specificity (lower is better) and generalization (lower is better) graphs against the number of modes of variation. These are also very similar between the two approaches.

Fig. 7.

(a) Surface-to-surface distance on best, median, and worst held-out samples. (b) Surface-to-surface distance plot between DeepSSM and Image2SSM. We observe that Image2SSM performs well compared to DeepSSM, but still fails to capture major outliers.