Artificial intelligence for MRI diagnosis of joints: a scoping review of the current state-of-the-art of deep learning-based approaches

Abstract

Deep learning-based MRI diagnosis of internal joint derangement is an emerging field of artificial intelligence, which offers many exciting possibilities for musculoskeletal radiology. A variety of investigational deep learning algorithms have been developed to detect anterior cruciate ligament tears, meniscus tears, and rotator cuff disorders. Additional deep learning-based MRI algorithms have been investigated to detect Achilles tendon tears, recurrence prediction of musculoskeletal neoplasms, and complex segmentation of nerves, bones, and muscles. Proof-of-concept studies suggest that deep learning algorithms may achieve similar diagnostic performances when compared to human readers in meta-analyses; however, musculoskeletal radiologists outperformed most deep learning algorithms in studies including a direct comparison. Earlier investigations and developments of deep learning algorithms focused on the binary classification of the presence or absence of an abnormality, whereas more advanced deep learning algorithms start to include features for characterization and severity grading. While many studies have focused on comparing deep learning algorithms against human readers, there is a paucity of data on the performance differences of radiologists interpreting musculoskeletal MRI studies without and with artificial intelligence support. Similarly, studies demonstrating the generalizability and clinical applicability of deep learning algorithms using realistic clinical settings with workflow-integrated deep learning algorithms are sparse. Contingent upon future studies showing the clinical utility of deep learning algorithms, artificial intelligence may eventually translate into clinical practice to assist detection and characterization of various conditions on musculoskeletal MRI exams.

Keywords: Artificial intelligence; Computer; Deep learning; Joints; Magnetic resonance imaging; Musculoskeletal system; Neural networks.

Fig. 1

Deep learning algorithm evaluation of the anterior cruciate ligament. A A 46-year-old woman who sustained an acute knee injury during tennis. Sagittal fat-suppressed proton density-weighted MR image shows an arthroscopy-confirmed full-thickness anterior cruciate ligament tear (arrow), which was correctly diagnosed by the deep learning algorithm (true positive). B A 40-year-old man who sustained an acute knee injury during ice hockey. Sagittal fat-suppressed proton density-weighted MR image shows an arthroscopy-confirmed intact anterior cruciate ligament (arrow), which was correctly diagnosed by the deep learning algorithm (true negative). C A 40-year-old man with chronic knee pain. Sagittal fat-suppressed proton density-weighted MR image shows mucoid degeneration of an intact anterior cruciate ligament (arrow), which was erroneously diagnosed as a tear by the deep learning algorithm (false positive). D A 19-year-old man with acute knee pain after fall. Sagittal fat-suppressed proton density-weighted MR image shows a full-thickness anterior cruciate ligament tear with a displaced bucket-handle tear of the lateral meniscus, resembling a double posterior cruciate ligament sign. The displaced meniscus fragment in the intercondylar notch (arrow) may be the underlying reason for the CNN assessing the anterior cruciate ligament erroneously as intact (false negative). Data were derived with a deep learning algorithm described in a study published by Germann et al. [4]

Fig. 2

Comparative performances of AI and human readers for MRI diagnosis of anterior cruciate ligament tears. Plots show the diagnostic performances of deep learning (DL) algorithms (A), musculoskeletal radiologists participating in artificial intelligence (AI) studies (B), and meta-analyses of human readers (C) for anterior cruciate ligament tear detection. The solid dots indicate the estimates of sensitivities (y-axis) and specificities (x-axis). The surrounding ellipses represent the corresponding 95% confidence intervals. Most studies are located exclusively in the left upper zone (white background), indicating at least acceptable diagnostic performance for diagnosis [62]. Right lower cut-out boxes represent a magnification of the left upper area (dashed box). In A and B: dark gray = Liu et al. [3]; yellow = Germann et al. [4]; blue = Namiri et al. [14]; green = Bien et al. [11]. In C: gray = Oei et al. [18]; orange = Smith et al. [20]; light blue = Phelan et al. [21]; red = Crawford et al. [19]. Note: Only studies reporting 95% CI were included. Test data set rules, settings, reference standards, and experience levels of readers differed between studies, which may limit the direct comparability of diagnostic performances

Fig. 3

MRI of the left knee joint in a 59-year-old patient with chronic medial knee pain. A Coronal short tau inversion recovery (STIR) MR image through the mid-body segment of the medial meniscus shows a horizontal cleavage tear (arrow). B Sagittal proton density-weighted MR image with spectral fat suppression shows the horizontal meniscus tear extending to the posterior horn of the medial meniscus (arrow). C AI-based assessment of the medial and lateral menisci predicted a medial meniscus tear with a probability of 84%. The heat map located the tear (colored area) correctly to the mid-body segment and junction to the posterior horn (C). Arthroscopic knee surgery confirmed the meniscus tear. Data were derived with a deep learning algorithm described in a study published by Fritz et al. [32]

Fig. 4

Comparative performances of AI (red and gray circles) and human readers (yellow and blue circles) for MRI diagnosis of meniscus tears. Plots show the diagnostic performances of selected deep learning (DL) algorithms and meta-analyses of human readers for MRI-based diagnosis of medial (A) and lateral (B) meniscus tears. The solid dots indicate the estimates of sensitivities (y-axis) and specificities (x-axis). The surrounding ellipses represent the corresponding 95% confidence intervals. For medial meniscus tears (A), all studies are located exclusively in the left upper zone (white background), indicating at least acceptable diagnostic performance for diagnosis [62]. For lateral meniscus tears, the performance estimates of the two DL studies of Fritz et al. and Rizk et al. occupy the left lower zone (light gray background), indicating a limited sensitivity for clinical application. Right lower cut-out boxes represent a magnification of the left upper area (dashed box). In A and B: gray = DL algorithm of Fritz et al. [32]; red = DL algorithm of Rizk et al. [33]; yellow = meta-analysis of Phelan et al. [21]; blue = meta-analysis of Oei et al. [18]. Note: Only studies reporting 95% CI and differentiating between the medial and lateral meniscus were included. Test data set rules, settings, reference standards, and experience levels of readers differed between studies, which may limit the direct comparability of diagnostic performances

Fig. 5

Sagittal T1-weighted MR image of the right shoulder (A), with manual (B) and AI-based (C) segmentations of the subscapularis (blue overlay), supraspinatus (red overlay), and infraspinatus/teres minor (yellow overlay) muscles. The manual (B) and AI-based (C) segmentations had high similarity with a Dice score > 0.93. The segmentation was derived with an AI-based algorithm described by Medina et al. [49]. Images courtesy of Martin Torriani M.D., Harvard Medical School, Massachusetts General Hospital, Boston, MA

Fig. 6

Artificial intelligence (AI) workflow as deployed at our institution. After acquiring the MRI study, the digital imaging and communications in medicine (DICOM) images are sent to the picture archiving and communication system (PACS). From the PACS, DICOM images can be routed to the local AI server either manually or based on the fulfillment of predefined criteria, such as DICOM header descriptions. After processing, the AI server sends the report as a PDF document back to the PACS

Fig. 7

Artificial intelligence (AI)-augmented knee MRI interpretation using an investigational AI algorithm. The AI report appears as part of the MRI study in the left column (yellow-framed arrow) and can be displayed in a viewport (lower right viewport) or separate window (not shown). In this patient, the AI algorithm predicted internal degeneration of the medial meniscus with a probability of 52% (red-framed arrow), based on the intrasubstance signal hyperintensities (white-framed arrows), as well as absent meniscus tear, meniscus extrusion (subluxation), meniscus ganglions cyst, anterior cruciate ligament tear, and medial collateral ligament tear