DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images

Abstract

Statistical shape modeling is an important tool to characterize variation in anatomical morphology. Typical shapes of interest are measured using 3D imaging and a subsequent pipeline of registration, segmentation, and some extraction of shape features or projections onto some lower-dimensional shape space, which facilitates subsequent statistical analysis. Many methods for constructing compact shape representations have been proposed, but are often impractical due to the sequence of image preprocessing operations, which involve significant parameter tuning, manual delineation, and/or quality control by the users. We propose DeepSSM: a deep learning approach to extract a low-dimensional shape representation directly from 3D images, requiring virtually no parameter tuning or user assistance. DeepSSM uses a convolutional neural network (CNN) that simultaneously localizes the biological structure of interest, establishes correspondences, and projects these points onto a low-dimensional shape representation in the form of PCA loadings within a point distribution model. To overcome the challenge of the limited availability of training images with dense correspondences, we present a novel data augmentation procedure that uses existing correspondences on a relatively small set of processed images with shape statistics to create plausible training samples with known shape parameters. In this way, we leverage the limited CT/MRI scans (40-50) into thousands of images needed to train a deep neural net. After the training, the CNN automatically produces accurate low-dimensional shape representations for unseen images. We validate DeepSSM for three different applications pertaining to modeling pediatric cranial CT for characterization of metopic craniosynostosis, femur CT scans identifying morphologic deformities of the hip due to femoroacetabular impingement, and left atrium MRI scans for atrial fibrillation recurrence prediction.

Fig. 1.

Illustration of how DeepSSM is trained and used for getting shape descriptors for shape analysis directly from images.

Fig.2.

Top: Shapes given by the original input CT scans (red dots) are augmented by sampling in the PDM shape space, from a normal distribution. Bottom-right: The resulting correspondences are used to transform original images of nearby samples (with a TPS warp) to create new images with known shape parameters.

Fig. 3.

The architecture of the CNN network.

Fig. 4.. Shape reconstruction errors in mm.

Each boxplot shows the error per-point per-shape in each category, for training, testing, and validation datasets. As ground truth correspondences are available, the error is simple Euclidean distance (in mm). For the unseen normal (unseen-N) and unseen pathological (unseen-P), the error is the minimum projection distance of the predicted point to original surface mesh (again in mm). (a) Metopic Craniosynostosis data, (b) Femur data, and (c) Left Atrium data.

Fig. 5.

(Left) CT scan of a normal head. (Middle) CT scan of a head shape affected by metopic craniosynostosis. (Right) The histograms of Mahalanobis distance for training, validation and testing datasets and the metopic-heads dataset (yellow bars, no data augmentation was performed).

Fig. 6.

(a) Schematics of cam-FAI. Normal femur (b) compared to a cam femur (c); circles show location of deformity. (d) cam FAI patient post-surgery. Surgical treatment aims to remove bony deformities.

Fig. 7.

The left two rows represent the images (seen normal femur, unseen normal femur and unseen pathological femur) and the corresponding shape reconstruction error (Hausdorff distance in mm) interpolated as a heatmap on the original meshes. On the right: [A] mean error of unseen normal femurs overlayed on mean shape, [B] mean error of unseen pathological femurs overlayed on mean shape, [C] standard deviation of error of unseen normal femurs overlayed on mean shape, [D] standard deciation of error of unseen pathological femurs overlayed on mean shape

Fig. 8.

Bottom row represent representative sample of different image types in our database, going from worst to best (left to right). Top row represent shape derived from corresponding image using the proposed method, with a distance map overlay from particle modeling shape reconstruction. (Right) S: Seen Data U: Unseen Data : Boxplot for AF recurrence probability difference using PCA loadings using the PDM directly and those estimated by DeepSSM