Machine Learning-Aided Diagnosis Enhances Human Detection of Perilunate Dislocations

Abstract

Background: Perilunate/lunate injuries are frequently misdiagnosed. We hypothesize that utilization of a machine learning algorithm can improve human detection of perilunate/lunate dislocations.

Methods: Participants from emergency medicine, hand surgery, and radiology were asked to evaluate 30 lateral wrist radiographs for the presence of a perilunate/lunate dislocation with and without the use of a machine learning algorithm, which was used to label the lunate. Human performance with and without the machine learning tool was evaluated using sensitivity, specificity, accuracy, and F1 score.

Results: A total of 137 participants were recruited, with 55 respondents from emergency medicine, 33 from radiology, and 49 from hand surgery. Thirty-nine participants were attending physicians or fellows, and 98 were residents. Use of the machine learning tool improved specificity from 88% to 94%, accuracy from 89% to 93%, and F1 score from 0.89 to 0.92. When stratified by training level, attending physicians and fellows had an improvement in specificity from 93% to 97%. For residents, use of the machine learning tool resulted in improved accuracy from 86% to 91% and specificity from 86% to 93%. The performance of surgery and radiology residents improved when assisted by the tool to achieve similar accuracy to attendings, and their assisted diagnostic performance reaches levels similar to that of the fully automated artificial intelligence tool.

Conclusions: Use of a machine learning tool improves resident accuracy for radiographic detection of perilunate dislocations, and improves specificity for all training levels. This may help to decrease misdiagnosis of perilunate dislocations, particularly when subspecialist evaluation is delayed.

Keywords: artificial intelligence; computer vision; deep learning; lunate dislocation; perilunate dislocation.

Figure 1.

Outline of machine learning algorithm previously developed for identification and automatic labeling of the lunate on lateral wrist radiographs.

Figure 2.

(a) Sample lateral wrist radiograph presented to participants for diagnosis of presence or absence of perilunate/lunate dislocation. (b) Sample lateral wrist radiograph with aid of machine learning algorithm to annotate the lunate position with a bounding box.

Figure 3.

Unassisted (baseline) and machine learning–assisted clinician performance in the study compared with overall performance of automated image classifier neural network algorithm. Note. Clinician performance displayed as mean with 95% confidence interval. Dashed line indicates receiver operating characteristic curve of the underlying algorithm as previously published, with an associated area under the curve of 0.986, demonstrating performance of the algorithm across all classification thresholds. “X” marker indicates automated classifier performance at a classification threshold of 0.5, with sensitivity of 93.8% and specificity of 93.3%. All points above the curve represent clinician performance superior to that of the automated classifier model, and points below the curve represent performance inferior to that of the algorithm. EM = emergency medicine.