Myeloma Spine and Bone Damage Score (MSBDS) on Whole-Body Computed Tomography (WBCT): Multiple Reader Agreement in a Multicenter Reliability Study

Abstract

Objective: To assess the reliability of the myeloma spine and bone damage score (MSBDS) across multiple readers with different levels of expertise and from different institutions. Methods: A reliability exercise, including 104 data sets of static images and complete CT examinations of patients affected by multiple myeloma (MM), was performed. A complementary imaging atlas provided detailed examples of the MSBDS scores, including low-risk and high-risk lesions. A total of 15 readers testing the MSBDS were evaluated. ICC estimates and their 95% confidence intervals were calculated based on mean rating (k = 15), absolute agreement, a two-way random-effects model and Cronbach's alpha. Results: Overall, the ICC correlation coefficient was 0.87 (95% confidence interval: 0.79-0.92), and the Cronbach's alpha was 0.93 (95% confidence interval: 0.94-0.97). Global inter- and intra-observer agreement among the 15 readers with scores below or equal to 6 points and scores above 6 points were 0.81 (95% C.I.: 0.72-0.86) and 0.94 (95% C.I.:0.91-0.98), respectively. Conclusion: We present a consensus-based semiquantitative scoring systems for CT in MM with a complementary CT imaging atlas including detailed examples of relevant scoring techniques. We found substantial agreement among readers with different levels of experience, thereby supporting the role of the MSBDS for possible large-scale applications. Significance and Innovations • Based on previous work and definitions of the MSBDS, we present real-life reliability data for quantitative bone damage assessment in multiple myeloma (MM) patients on CT. • In this study, reliability for the MSBDS, which was tested on 15 readers with different levels of expertise and from different institutions, was shown to be moderate to excellent. • The complementary CT imaging atlas is expected to enhance unified interpretations of the MSBDS between different professionals dealing with MM patients in their routine clinical practice.

Keywords: bone; computed tomography; multiple myeloma; quantitative imaging.

Figure 1

Scoring bone damage and instability: spectrum of findings. (A) Focal lytic lesions >5 mm in diameter located at the left sacrum (white arrows). In this case the MSBDS was 2 (1 + 1). (B) Single focal lytic lesion >5 mm in the vertebral body (white arrow) with no vertebral collapse (sagittal not shown). The adjacent smaller focal lytic lesion (green line) is <5 mm (no points in the MSBDS). In this case, the MSBDS was 1. (C) Large lytic lesion at the junctional spine (L5-S1) with cortical erosion, collapse/involvement >50%, posterolateral (facet, pedicle) involvement and more than 2/3 of bone diameter. In this case, the MSBDS was 11 (3 + 3 + 2 + 3): the lesion was considered “high-risk” and immediate surgical or radiation oncologist consultation was warranted. In this case, there was also possible spinal canal involvement. (D) Lytic lesion >5 mm (white arrow) at the junctional spine (thoracic spine) with collapse/involvement <50% and a small (small white arrow) focal lesion at the anterior arch of the right rib cage with extraosseous extension. In this case, the MSBDS was 6 (3 + 2 + 1): the lesion was considered “medium-risk” (5–10 with medium risk of pathologic fracture). (E) Large lytic lesion at the junctional spine (thoracic spine) with collapse/involvement >50%, posterolateral (facet, pedicle) involvement and more than 2/3 of bone diameter. In this case, the MSBDS was 11 (3 + 3 + 2 + 3): the lesion was considered “high-risk” and immediate surgical or radiation oncologist consultation was warranted. In this case, there is spinal canal involvement. (F) Lytic lesion at the left femoral neck (white arrow). This lesion alone warrants 5 points in the MSBDS: the lesion was considered “medium-risk”, although immediate fracture seems unlikely.

Figure 2

Frequency of bone lesions in the web-based reliability assessment. In this graph, the distribution of the degree of pathological findings, according to the MSBDS, is presented. The range of the MSBDSs was between 1 and 18. Most lesions (21%) were small lesions with an MSBDS of 1.

Figure 3

Scoring bone damage and instability: spectrum of findings with maximum disagreement among readers. Discrepancies with an MSBDS > 6. (A) Focal lytic lesions >5 mm in diameter located at the left sacrum (white arrow) with another small lesion in the sacrum near the sacroiliac joint. In this case, the MSBDS was 7 with a standard deviation of 4.9. (B) Single large focal lytic lesion >5 mm in the vertebral body (white arrow) with no vertebral collapse (sagittal not shown) but possible spinal canal infiltration. In this case, the MSBDS was 11 with a standard deviation of 5.3 due to difficulties in spinal canal assessment mainly by non-specialists. In these cases, a sub-specialized second reading should be recommended. Discrepancies with an MSBDS < 6. (C) Multiple lytic lesions >5 mm (white arrows and the black arrow) at the junctional spine (T11-L1 level) with involvement of the vertebral body and pedicle. In this case, the MSBDS was 4 with a standard deviation of 2.6. (D) Single large focal lytic lesion >5 mm in the vertebral body (white arrow) with no vertebral collapse. In this case, the MSBDS was 5.7 with a standard deviation of 3.3 due to difficulties in spinal canal assessment, bone diameter and extraosseous involvement (asterisks).