Deep Convolutional Neural Network-Based Diagnosis of Anterior Cruciate Ligament Tears: Performance Comparison of Homogenous Versus Heterogeneous Knee MRI Cohorts With Different Pulse Sequence Protocols and 1.5-T and 3-T Magnetic Field Strengths

Abstract

Objectives: The aim of this study was to clinically validate a Deep Convolutional Neural Network (DCNN) for the detection of surgically proven anterior cruciate ligament (ACL) tears in a large patient cohort and to analyze the effect of magnetic resonance examinations from different institutions, varying protocols, and field strengths.

Materials and methods: After ethics committee approval, this retrospective analysis of prospectively collected data was performed on 512 consecutive subjects, who underwent knee magnetic resonance imaging (MRI) in a total of 59 different institutions followed by arthroscopic knee surgery at our institution. The DCNN and 3 fellowship-trained full-time academic musculoskeletal radiologists evaluated the MRI examinations for full-thickness ACL tears independently. Surgical reports served as the reference standard. Statistics included diagnostic performance metrics, including sensitivity, specificity, area under the receiver operating curve ("AUC ROC"), and kappa statistics. P values less than 0.05 were considered to represent statistical significance.

Results: Anterior cruciate ligament tears were present in 45.7% (234/512) and absent in 54.3% (278/512) of the subjects. The DCNN had a sensitivity of 96.1%, which was not significantly different from the readers (97.5%-97.9%; all P ≥ 0.118), but significantly lower specificity of 93.1% (readers, 99.6%-100%; all P < 0.001) and "AUC ROC" of 0.935 (readers, 0.989-0.991; all P < 0.001) for the entire cohort. Subgroup analysis showed a significantly lower sensitivity, specificity, and "AUC ROC" of the DCNN for outside MRI (92.5%, 87.1%, and 0.898, respectively) than in-house MRI (99.0%, 94.4%, and 0.967, respectively) examinations (P = 0.026, P = 0.043, and P < 0.05, respectively). There were no significant differences in DCNN performance for 1.5-T and 3-T MRI examinations (all P ≥ 0.753, respectively).

Conclusions: Deep Convolutional Neural Network performance of ACL tear diagnosis can approach performance levels similar to fellowship-trained full-time academic musculoskeletal radiologists at 1.5 T and 3 T; however, the performance may decrease with increasing MRI examination heterogeneity.

FIGURE 1

Illustration of the deep learning-based algorithm. Top box: First, a preprocessing step selects, rescales, and crops coronal and sagittal fluid-sensitive fat-suppressed MRI scans. Middle box: Second, the coronal and sagittal MRI scans are processed independently in parallel and then concatenated before being processed by one dense layer. Bottom box: Finally, one softmax layer extracted the confidence level for an anterior cruciate ligament (ACL) tear.

FIGURE 2

Flowchart of the study design and subjects. ACL, anterior cruciate ligament; DCNN, Deep Convolutional Neural Network.

FIGURE 3

MRI of the left knee of a 41-year-old woman with knee injury 1 week earlier. Sagittal intermediate-weighted turbo spin echo image with fat suppression (A) and coronal short tau inversion recovery image (B) show a full-thickness tear of the midsubstance of the ACL (arrows), which was confirmed by arthroscopic surgery. The DCNN and all 3 radiologists correctly diagnosed the full-thickness ACL tear.

FIGURE 4

MRI of the left knee of a 38-year-old woman with knee injury 3 months earlier. Coronal intermediate-weighted turbo spin echo MRI scan with fat suppression (A) and sagittal intermediate-weighted turbo spin echo MRI scans with fat suppression (B and C) show tearing of the anterior cruciate ligament with greater than 80% disruption of fibers (white arrows) and some intact fibers (black arrow) remaining, as confirmed by surgery. Two of the 3 radiologists classified the MRI scans as a partial-thickness tear (<80% of fiber discontinuity), representing a false-negative interpretation. One radiologist and the DCNN correctly diagnosed a full-thickness ACL tear, representing a true-positive interpretation.

FIGURE 5

MRI of the right knee of a 43-year-old woman with knee injury 7 days earlier. Sagittal intermediate-weighted turbo spin echo image with fat suppression (A) and coronal short tau inversion recovery image (B) show diffuse and focal (black arrow) signal hyperintensity of the anterior cruciate ligament (ACL) indicative of mucoid degeneration and an intraligamentous ganglion cyst (white arrows) with otherwise continuous fibers in normal oblique orientation. Arthroscopic surgery confirmed mucoid degeneration of the ACL without fiber discontinuity. All 3 radiologists correctly diagnosed an intact ACL, whereas the DCNN erroneously classified the ACL as torn, representing a false-positive case.