Automated detection of spinal bone marrow oedema in axial spondyloarthritis: training and validation using two large phase 3 trial datasets

Abstract

Objective: To evaluate the performance of machine learning (ML) models for the automated scoring of spinal MRI bone marrow oedema (BMO) in patients with axial spondyloarthritis (axSpA) and compare them with expert scoring.

Methods: ML algorithms using SpineNet software were trained and validated on 3483 spinal MRIs from 686 axSpA patients across two clinical trial datasets. The scoring pipeline involved (i) detection and labelling of vertebral bodies and (ii) classification of vertebral units for the presence or absence of BMO. Two models were tested: Model 1, without manual segmentation, and Model 2, incorporating an intermediate manual segmentation step. Model outputs were compared with those of human experts using kappa statistics, balanced accuracy, sensitivity, specificity and AUC.

Results: Both models performed comparably to expert readers, regarding presence vs absence of BMO. Model 1 outperformed Model 2, with an AUC of 0.94 (vs 0.88), accuracy of 75.8% (vs 70.5%) and kappa of 0.50 (vs 0.31) using absolute reader consensus scoring as the external reference; this performance was similar to the expert inter-reader accuracy of 76.8% and kappa of 0.47 in a radiographic axSpA dataset. In a non-radiographic axSpA dataset, Model 1 achieved an AUC of 0.97 (vs 0.91 for Model 2), accuracy of 74.6% (vs 70%) and kappa of 0.52 (vs 0.27), comparable to the expert inter-reader accuracy of 74.2% and kappa of 0.46.

Conclusion: ML software shows potential for automated MRI BMO assessment in axSpA, offering benefits such as improved consistency, reduced labour costs and minimized inter- and intra-reader variability.

Trial registration: Clinicaltrials.gov, http://clinicaltrials.gov, MEASURE 1 study (NCT01358175); PREVENT study (NCT02696031).

Keywords: MRI; ankylosing spondylitis; artificial intelligence; axial spondyloarthritis; bone marrow oedema; inflammation; machine learning; osteitis.

Figure 1.

Overall bone marrow oedema automated detection process. In the examples on the right, three slices of T1 and STIR images are displayed; however, our models process all the slices for a given volume unit, ensuring a thorough analysis and accurate representation of the entire anatomical structure

Figure 2.

Quantitative examples of MEASURE 1, including both the annotation by the expert radiologist and the model annotation prediction. First panel, STIR sequence; second panel, T1-weighted sequence; third panel, annotation by the expert in the STIR sequence; fourth panel, model annotation prediction in the STIR sequence