Automatic Spine Segmentation and Parameter Measurement for Radiological Analysis of Whole-Spine Lateral Radiographs Using Deep Learning and Computer Vision

Abstract

Radiographic examination is essential for diagnosing spinal disorders, and the measurement of spino-pelvic parameters provides important information for the diagnosis and treatment planning of spinal sagittal deformities. While manual measurement methods are the golden standard for measuring parameters, they can be time consuming, inefficient, and rater dependent. Previous studies that have used automatic measurement methods to alleviate the downsides of manual measurements showed low accuracy or could not be applied to general films. We propose a pipeline for automated measurement of spinal parameters by combining a Mask R-CNN model for spine segmentation with computer vision algorithms. This pipeline can be incorporated into clinical workflows to provide clinical utility in diagnosis and treatment planning. A total of 1807 lateral radiographs were used for the training (n = 1607) and validation (n = 200) of the spine segmentation model. An additional 200 radiographs, which were also used for validation, were examined by three surgeons to evaluate the performance of the pipeline. Parameters automatically measured by the algorithm in the test set were statistically compared to parameters measured manually by the three surgeons. The Mask R-CNN model achieved an average precision at 50% intersection over union (AP50) of 96.2% and a Dice score of 92.6% for the spine segmentation task in the test set. The mean absolute error values of the spino-pelvic parameters measurement results were within the range of 0.4° (pelvic tilt) to 3.0° (lumbar lordosis, pelvic incidence), and the standard error of estimate was within the range of 0.5° (pelvic tilt) to 4.0° (pelvic incidence). The intraclass correlation coefficient values ranged from 0.86 (sacral slope) to 0.99 (pelvic tilt, sagittal vertical axis).

Keywords: Computer vision; Computer-assisted diagnosis; Mask R-CNN; Spine.

Fig. 1

A flowchart illustrating an automated measurement pipeline, which consists of instance segmentation models and computer vision algorithms

Fig. 2

Overview of image data allocation for training the Mask R-CNN model

Fig. 3

Methods for measuring spinal parameters. a SVA was measured from the C7 and sacral vertebrae. b TK, TLK, and LL were measured in T4-T12, T10-L2, and L1-sacrum, respectively. c SS is the slope of the straight line through the left and top points (red points) of the sacral endplate, and PT was estimated from the line through the HA at the midpoint (green points) of the sacral endplate. d An example of upper and lower endplate slope measurement

Fig. 4

Block diagram of Mask R-CNN algorithm for spine segmentation (FM = feature maps; RPN = region proposal network; FCN = fully connected network; FC layer = fully connected layer)

Fig. 5

Vertebral body polygon approximation procedure of DP algorithm. a When ${\mathit{Dist}}_{\mathit{max}}$ ≥ $\epsilon$ from $l_1$, $P_{\mathit{start}}$ and $P_2$ are then used for the baseline. b ${\mathit{Dist}}_{\mathit{max}}$ is computed at the points between $P_{\mathit{start}}$ and $P_2$. When ${\mathit{Dist}}_{\mathit{max}}$ < $\epsilon$ from $l_2$, $P_2$ becomes the starting point. c When ${\mathit{Dist}}_{\mathit{max}}$ ≥ $\epsilon$ from $l_2$, $P_2$ and $P_3$ are used for the next baseline. d ${\mathit{Dist}}_{\mathit{max}}$ is calculated at the points between $P_2$ and $P_3$. When ${\mathit{Dist}}_{\mathit{max}}$ < $\epsilon$ from $l_2$, $P_3$ becomes the starting point. e, f The repetition of the above process

Fig. 6

Computer vision analysis results for predicted and ground truth masks. a Lateral X-ray images. b Ground truth images. c Predicted images of Mask R-CNN model. d Vertebral bodies in the T4-L5 vertebrae class are identified based on the central coordinates of the sacral. e The results of parameter measurement of the computer vision algorithm were visualized

Fig. 7

There were two types of failed inference results from the Mask R-CNN model on the test-set. a T4 segmentation failure image. b Case of segmentation failure where the sacrum and L5 overlap

Fig. 8

Bland–Altman plots: kyphosis and spino-pelvic parameters

Fig. 9

Vertebral segmentation results of Mask R-CNN in an untrained clinical case. a Spinal instruments. b Fractured vertebrae. c Bone cement