Muscle Cross-Sectional Area Segmentation in Transverse Ultrasound Images Using Vision Transformers

Abstract

Automatically measuring a muscle’s cross-sectional area is an important application in clinical practice that has been studied extensively in recent years for its ability to assess muscle architecture. Additionally, an adequately segmented cross-sectional area can be used to estimate the echogenicity of the muscle, another valuable parameter correlated with muscle quality. This study assesses state-of-the-art convolutional neural networks and vision transformers for automating this task in a new, large, and diverse database. This database consists of 2005 transverse ultrasound images from four informative muscles for neuromuscular disorders, recorded from 210 subjects of different ages, pathological conditions, and sexes. Regarding the reported results, all of the evaluated deep learning models have achieved near-to-human-level performance. In particular, the manual vs. the automatic measurements of the cross-sectional area exhibit an average discrepancy of less than 38.15 mm2, a significant result demonstrating the feasibility of automating this task. Moreover, the difference in muscle echogenicity estimated from these two readings is only 0.88, another indicator of the proposed method’s success. Furthermore, Bland−Altman analysis of the measurements exhibits no systematic errors since most differences fall between the 95% limits of agreements and the two readings have a 0.97 Pearson’s correlation coefficient (p < 0.001, validation set) with ICC (2, 1) surpassing 0.97, showing the reliability of this approach. Finally, as a supplementary analysis, the texture of the muscle’s visible cross-sectional area was examined using deep learning to investigate whether a classification between healthy subjects and patients with pathological conditions solely from the muscle texture is possible. Our preliminary results indicate that such a task is feasible, but further and more extensive studies are required for more conclusive results.

Keywords: cross-sectional area; deep learning; textural analysis; ultrasound; vision transformers.

Figure 1

Sample of ultrasound recordings with normal and high echogenicity. (A,E) shows images from the T.A., (B,F) shows images from the R.F., (C,G) shows images extracted from the GCM, and finally, (D,H) are images of the B.B.

Figure 2

Sample of ultrasound images with their corresponding annotation. The CSA area contours are depicted with yellow lines. The grayscale level (GSL) histogram and its mean value are extracted inside the CSA. (A) shows the measurements of T.A., (B) shows the measurements of R.F., (C) shows the measurements of the GCM, and (D) demonstrates B.B.

Figure 3

The process of extracting and isolating the CSA texture at each ultrasound recording.

Figure 4

The deep learning architectures used in this study to automatically segment the CSA. In (A), the original UNet [17] is depicted; in (B), the vision transformer TMUNet [22] is depicted; in (C), the Attention-UNet [19] model is depicted; in (D), the UNet++ [32] is depicted; and finally, in (E), the UNeXt [23] architecture is depicted.

Figure 5

In (A), the flowchart of the proposed pipeline to extract the CSA and its mean grey level (echogenicity) is presented. In (B), the two classification analyses from the predicted CSA texture are depicted.

Figure 6

The 5-fold evaluation protocol. At each iteration, 80% of the examinations were considered training sets and the remaining 20% were considered test sets. The average accuracy of all of the iterations was regarded as the model performance.

Figure 7

The Resnet18 classifier. For each image input, the classifier predicts which group the texture belongs to.

Figure 8

Qualitative results of the CSA segmentation for samples of the (A) T.A., (B) R.F., (C) GCM, and (D) B.B. The red areas are the predicted masks superimposed onto the input image.

Figure 9

Bland−Altman plots of the (A) CSA and (B) echogenicity in all the databases.

Figure 10

Bland−Altman plots of the CSA for the (A) T.A., (B) R.F., (C) GCM, and (D) B.B.

Figure 11

Bland−Altman plots of the CSA for (A) Group 1, (B) Group 2, and (C) Group 3.

Figure 12

Box plot diagrams of the automatic method performances in terms of (A) precision and (B) recall metrics, dividing the dataset between the groups.

Figure 13

Confusion matrices for the per-group analysis. In (A), the manually extracted CSA textures were used and in (B), the automatic were used.

Figure 14

Confusion matrices for the group classification for the (A) T.A., (B) R.F., (C) GCM, and (D) B.B.

Figure 15

Grad-CAM analysis for the group classification of each muscle section.

Figure 16

Confusion matrices for the young vs. elderly subjects’ analysis. In (A), the manually extracted CSA texture was used and in (B), the automatic was used.