A deep learning method for predicting knee osteoarthritis radiographic progression from MRI

Abstract

Background: The identification of patients with knee osteoarthritis (OA) likely to progress rapidly in terms of structure is critical to facilitate the development of disease-modifying drugs.

Methods: Using 9280 knee magnetic resonance (MR) images (3268 patients) from the Osteoarthritis Initiative (OAI) database , we implemented a deep learning method to predict, from MR images and clinical variables including body mass index (BMI), further cartilage degradation measured by joint space narrowing at 12 months.

Results: Using COR IW TSE images, our classification model achieved a ROC AUC score of 65%. On a similar task, trained radiologists obtained a ROC AUC score of 58.7% highlighting the difficulty of the classification task. Additional analyses conducted in parallel to predict pain grade evaluated by the WOMAC pain index achieved a ROC AUC score of 72%. Attention maps provided evidence for distinct specific areas as being relevant in those two predictive models, including the medial joint space for JSN progression and the intra-articular space for pain prediction.

Conclusions: This feasibility study demonstrates the interest of deep learning applied to OA, with a potential to support even trained radiologists in the challenging task of identifying patients with a high-risk of disease progression.

Fig. 1

Overview of the image preprocessing pipeline. The raw MR image is first re-oriented so that both left and right knees are similarly oriented. Noteworthy, only the left knee image is flipped, whereas the right is maintained as is, in order to obtain uniform orientations across the dataset. The N4 bias field correction is then applied, followed by a color normalization step

Fig. 2

Global overview of the feature extraction step. Converted images undergo several pre-processing steps (reorientation, N4 bias field correction, color normalization) before submitting each slice as input to a pre-trained EfficientNet-B0 network. This neural network will compute 1280 features (or numerical descriptors) for each slice, resulting in depth × 1280 features for an input volume (where depth corresponds to the number of input slices)

Fig. 3

Global overview of the model. The purple-shaded area is a first sub-model aiming to locate regions of interest within input images. The green-shaded area represents the classification sub-model, which aggregates both image and clinical information into progression (or pain score) probabilities

Fig. 4

ROC curves and confusion matrix of the binary classification model to identify 12-month OA progressors. The model aims to identify knees for which JSN (t + 12 months) ≤ − 0.5 mm. The five curves correspond to the fivefold cross-validation scheme. The dotted diagonal line (purple) illustrates the performance of a random predictor

Fig. 5

ROC curve and confusion matrix for prediction of pain severity. Graphic representations are shown for the binary task of classifying sets of pain scores across the cross-validation process

Fig. 6

Visual interpretation of relevant zones identified by prediction models. The upper row corresponds to prediction of JSN progression and the bottom row to pain prediction. Yellow areas are the ones considered of high interest by the model: the more intense the yellow, the higher its contribution to a high score for JSN progression prediction (bottom row, coronal view) or severe pain classification (top row, coronal view). All images are obtained from patient 9932578 (right knee)