Validation of musculoskeletal segmentation model with uncertainty estimation for bone and muscle assessment in hip-to-knee clinical CT images

Abstract

Deep learning-based image segmentation has allowed for the fully automated, accurate, and rapid analysis of musculoskeletal (MSK) structures from medical images. However, current approaches were either applied only to 2D cross-sectional images, addressed few structures, or were validated on small datasets, which limit the application in large-scale databases. This study aimed to validate an improved deep learning model for volumetric MSK segmentation of the hip and thigh with uncertainty estimation from clinical computed tomography (CT) images. Databases of CT images from multiple manufacturers/scanners, disease status, and patient positioning were used. The segmentation accuracy, and accuracy in estimating the structures volume and density, i.e., mean HU, were evaluated. An approach for segmentation failure detection based on predictive uncertainty was also investigated. The model has improved all segmentation accuracy and structure volume/density evaluation metrics compared to a shallower baseline model with a smaller training database (N = 20). The predictive uncertainty yielded large areas under the receiver operating characteristic (AUROC) curves (AUROCs ≥ .95) in detecting inaccurate and failed segmentations. Furthermore, the study has shown an impact of the disease severity status on the model's predictive uncertainties when applied to a large-scale database. The high segmentation and muscle volume/density estimation accuracy and the high accuracy in failure detection based on the predictive uncertainty exhibited the model's reliability for analyzing individual MSK structures in large-scale CT databases.

Fig. 1

Segmentation labels of the bones and muscles

Fig. 2

Overall scheme for validation of musculoskeletal segmentation model for automated assessment of bones and muscles in CT images with uncertainty estimation

Fig. 3

Summary of the research questions tackled in the study with the corresponding databases and methodologies used in the experiments. Sect: section numbers in the paper, ROI: region-of-interest, Hip OA: hip osteoarthritis, GT: ground-truth (annotation), DB: database, N: number of cases

Fig. 4

Distributions of the segmentation accuracy (a), predictive uncertainty (b), and volume/mean HU accuracy (c) of the bones and muscles (averaged on all structures) by each model applied to DB#1 (N = 50). Horizontal lines in the boxes represent the medians, while blue boxes represent the means. Detailed values are depicted in Supplementary Figs. A.1-A.5. DC: Dice coefficient, ASD: average symmetric surface distance, AVE: average volume error, AIE: average intensity error, n.s.: not significant, *: p < 0.017, Student’s t-test or Wilcoxon signed rank sum test with Bonferroni correction

Fig. 5

Receiver operating characteristic (ROC) curves of the inaccurate and failed segmentation detection in DB#1 (N = 50) using the predictive uncertainty. Thresholds were determined based on the median absolute deviations (σ) of the DC

Fig. 6

Distributions of the accuracy evaluation metrics and predictive uncertainty of the three MSK structure groups, i.e., thigh (left) and hip (middle) muscles and bones (right), in terms of the disease status of body sides in hip OA patients in internal validation DB#1 (a) and large-scale predictive uncertainty analysis in DB#5) (b). N: number of cases. n.s.: not significant, *: p < 0.004. (Based on Shapiro’s normality test, the hypothesis test was performed using either the Wilcoxon signed-rank test or the Student’s t-test. Bonferroni correction was used for the multiple comparisons.)

Fig. 7

Comparison between segmentation model accuracy (a, c) and predictive uncertainty (b) of the GMED muscle in the multi-manufacturer/scanner databases DB#1(N = 50), DB2(N = 18), DB#3(N = 10) and DB#4(N = 20). DC: Dice coefficient, ASD: Average symmetric surface distance, AVE: Average volume error, AIE: Average intensity error, su: supine, st: standing, n.s.: not significant, *: p < 0.01. (Based on Shapiro’s normality test, the hypothesis tests were performed using either the Wilcoxon signed-rank test or the Student’s t-test with Bonferroni correction). The triangles indicate the cases corresponding to the 5th (blue filled triangle) and 95th (red filled upside down triangle) quantiles of the predictive uncertainty visualized in A.7 and A.8.

Fig. 8

Ground-truth (GT) and predicted (Auto) segmentations of the unaffected (Un.) and affected (Aff.) sides of a representative hip OA case (median DC in Fig. 4) with diagnostic biomarkers, histograms, and muscle density visualizations of the gluteus maximus (GMAX) and gluteus medius (GMED) muscles.