Automatic segmentation of lower limb muscles from MR images of post-menopausal women based on deep learning and data augmentation

Abstract

Individual muscle segmentation is the process of partitioning medical images into regions representing each muscle. It can be used to isolate spatially structured quantitative muscle characteristics, such as volume, geometry, and the level of fat infiltration. These features are pivotal to measuring the state of muscle functional health and in tracking the response of the body to musculoskeletal and neuromusculoskeletal disorders. The gold standard approach to perform muscle segmentation requires manual processing of large numbers of images and is associated with significant operator repeatability issues and high time requirements. Deep learning-based techniques have been recently suggested to be capable of automating the process, which would catalyse research into the effects of musculoskeletal disorders on the muscular system. In this study, three convolutional neural networks were explored in their capacity to automatically segment twenty-three lower limb muscles from the hips, thigh, and calves from magnetic resonance images. The three neural networks (UNet, Attention UNet, and a novel Spatial Channel UNet) were trained independently with augmented images to segment 6 subjects and were able to segment the muscles with an average Relative Volume Error (RVE) between -8.6% and 2.9%, average Dice Similarity Coefficient (DSC) between 0.70 and 0.84, and average Hausdorff Distance (HD) between 12.2 and 46.5 mm, with performance dependent on both the subject and the network used. The trained convolutional neural networks designed, and data used in this study are openly available for use, either through re-training for other medical images, or application to automatically segment new T1-weighted lower limb magnetic resonance images captured with similar acquisition parameters.

Fig 1

Comparison of thigh MRI for young healthy individuals (A, B, C) and older individuals (D, E, F).

Fig 2. The process of generating the inputs for the neural network.

The MRI (example thigh 2D slice) inputted into the neural network (A) was manually segmented (B) and the segmentations were transformed into a labelled image (C). The greyscale colours of each muscle were ordered from 1–37 as they appear alphabetically.

Fig 3. The Spatial channel UNet (SC-UNet).

The spatial location of the input image was first calculated. Spatial information and the image were inputted into the network simultaneously, where they were split instantly. The image data went through the standard UNet architecture, while the spatial channel took an integer, p∈[0,100], and activated the p^th node of the input to the fully connected linear layer. Each input node was connected to each output node, of which there were 37 (equal to the number of muscles), allowing the locations of the muscles to be learned along the longitudinal axis.

Fig 4. Training (solid) and validation (dashed) loss curves calculated throughout the training phase for the UNet (orange), A-UNet (green), and SC-UNet (blue).

The cross-entropy loss shown was calculated for each batch of training or validation data and averaged across each epoch.

Fig 5. The RVE (%), DSC, and HD (mm) error metrics calculated through comparison of the reference and automatically generated segmentations for one testing subject across the three models tested (* p<0.05, ** p<0.01).

Fig 6. The RVE (%), DSC, and HD (mm) for the 23 muscles of the same testing subject as in Fig 5, calculated using each of the three CNN models tested, both with and without augmented images included in the training phase.

Significant differences are reported with *(p<0.05) and ** (p<0.01).

Fig 7

RVE (%), DSC, and HD (mm) calculated from the segmentations of 23 muscles from 6 different testing subjects for the UNet (left column), A-UNet (middle column) and SC-UNet (right column). Connections between distributions highlight significant differences between the mean error of the connected distributions. The boxplots with longer lines pointing at them had significant difference when compared to those with shorter lines pointing at them (* p <0.05, ** p<0.01, *** p<0.001). The three error metrics found for each subject across the three CNN models were also compared. A green boxplot highlights the statistically optimal result for a particular subject with significant improvement from the other two models (p<0.05). A yellow boxplot highlights the statistically best results for a particular subject with significant improvement from one of the other two models (p<0.05). Note that S0 is the initial test subject used in test 2).

Fig 8. Visual representation of ground truth and automatic segmentations outputted from the best performing model, the UNet trained with the augmented imaging datasets.

Three slices taken from halfway along the shank, thigh, and hip sections are presented, with the manual and automatic segmentations shown on the left and right, respectively. Each row of images corresponds to each of the six segmented subjects. Yellow arrows highlight regions that are well segmented, whereas purple and blue arrows highlight regions with segmentation inaccuracies.

Fig 9

Radar plots showing the absolute RVE (%, upper left hand corner), DSC (upper right hand corner), and HD (mm, lower centre) for the segmentation accuracy found for each muscle, averaged across the 6 tested subjects. The UNet, A-UNet, and SC-UNet appear in orange, green, and blue respectively, with the pink area of the graph representing the errors occurring through inter-operator repeatability. The three locations of the muscles: those within the hips, thigh, and calf, are highlighted with the solid, dashed and dotted black line along the circumference of the radar plots, respectively. The muscles are abbreviated according to Table 1.