Deep Learning Approach for Evaluating Knee MR Images: Achieving High Diagnostic Performance for Cartilage Lesion Detection

Abstract

Purpose To determine the feasibility of using a deep learning approach to detect cartilage lesions (including cartilage softening, fibrillation, fissuring, focal defects, diffuse thinning due to cartilage degeneration, and acute cartilage injury) within the knee joint on MR images. Materials and Methods A fully automated deep learning-based cartilage lesion detection system was developed by using segmentation and classification convolutional neural networks (CNNs). Fat-suppressed T2-weighted fast spin-echo MRI data sets of the knee of 175 patients with knee pain were retrospectively analyzed by using the deep learning method. The reference standard for training the CNN classification was the interpretation provided by a fellowship-trained musculoskeletal radiologist of the presence or absence of a cartilage lesion within 17 395 small image patches placed on the articular surfaces of the femur and tibia. Receiver operating curve (ROC) analysis and the κ statistic were used to assess diagnostic performance and intraobserver agreement for detecting cartilage lesions for two individual evaluations performed by the cartilage lesion detection system. Results The sensitivity and specificity of the cartilage lesion detection system at the optimal threshold according to the Youden index were 84.1% and 85.2%, respectively, for evaluation 1 and 80.5% and 87.9%, respectively, for evaluation 2. Areas under the ROC curve were 0.917 and 0.914 for evaluations 1 and 2, respectively, indicating high overall diagnostic accuracy for detecting cartilage lesions. There was good intraobserver agreement between the two individual evaluations, with a κ of 0.76. Conclusion This study demonstrated the feasibility of using a fully automated deep learning-based cartilage lesion detection system to evaluate the articular cartilage of the knee joint with high diagnostic performance and good intraobserver agreement for detecting cartilage degeneration and acute cartilage injury. © RSNA, 2018 Online supplemental material is available for this article .

Figure 1:

The convolutional neural network (CNN) architecture for the deep learning–based cartilage lesion detection system. The proposed method consisted of segmentation and classification CNNs that were connected in a cascaded fashion to create a fully automated image processing pipeline. BN = batch normalization, ReLu = rectified-linear activation, 2D = two-dimensional.

Figure 2:

Images show application of the customized software program used by the musculoskeletal radiologist and musculoskeletal radiology fellows to determine the presence or absence of a cartilage lesion in each image patch on each image section. From left to right, the images include a T2 map (in milliseconds), a proton density–weighted fast spin-echo (PD-FSE) MR image, and a fat-suppressed T2-weighted fast spin-echo ( T2-FSE) MR image. The annotation panel on the right was used for selecting the image patches and recording the image interpretations. The small image on the bottom right shows the automatically generated image patches on the articular surfaces of the femur (yellow boxes) and tibia (blue boxes). The yellow box on the fat-suppressed T2-weighted fast spin-echo image shows the current selected image patch and its location on the articular surface.

Figure 3a:

Sagittal fat-suppressed T2-weighted fast spin-echo MR images of segmented cartilage show image patches with (a) cartilage softening on the lateral tibial plateau, (b) cartilage fissuring on the medial femoral condyle, (c) focal cartilage defect on the medial femoral condyle, and (d) diffuse cartilage thinning on the lateral femoral condyle that were correctly identified by the cartilage lesion detection system (arrow).

Figure 3b:

Sagittal fat-suppressed T2-weighted fast spin-echo MR images of segmented cartilage show image patches with (a) cartilage softening on the lateral tibial plateau, (b) cartilage fissuring on the medial femoral condyle, (c) focal cartilage defect on the medial femoral condyle, and (d) diffuse cartilage thinning on the lateral femoral condyle that were correctly identified by the cartilage lesion detection system (arrow).

Figure 3c:

Sagittal fat-suppressed T2-weighted fast spin-echo MR images of segmented cartilage show image patches with (a) cartilage softening on the lateral tibial plateau, (b) cartilage fissuring on the medial femoral condyle, (c) focal cartilage defect on the medial femoral condyle, and (d) diffuse cartilage thinning on the lateral femoral condyle that were correctly identified by the cartilage lesion detection system (arrow).

Figure 3d:

Sagittal fat-suppressed T2-weighted fast spin-echo MR images of segmented cartilage show image patches with (a) cartilage softening on the lateral tibial plateau, (b) cartilage fissuring on the medial femoral condyle, (c) focal cartilage defect on the medial femoral condyle, and (d) diffuse cartilage thinning on the lateral femoral condyle that were correctly identified by the cartilage lesion detection system (arrow).

Figure 4:

Receiver operating characteristic (ROC) curves show the diagnostic performance of the cartilage lesion detection system for detecting cartilage lesions within the knee joint. Solid lines = ROC curves for the two individual evaluations performed by the cartilage lesion detection system. Dashed line = diagonal line, with an area under the ROC curve (AUC) of 0.5. The AUCs of the cartilage lesion detection system were 0.917 and 0.914 for evaluations 1 and 2, respectively, both indicating high overall diagnostic accuracy. Sensitivity and specificity for the radiology residents, musculoskeletal radiology fellows, musculoskeletal radiologist, and evaluations 1 and 2 of the cartilage lesion detection system at the optimal threshold of the Youden index are also plotted. Note that the sensitivity and specificity of the clinical radiologists are in close proximity to the ROC curves of the cartilage lesion detection system.