Artificial intelligence for automated detection and measurements of carpal instability signs on conventional radiographs

Abstract

Objectives: To develop and validate an artificial intelligence (AI) system for measuring and detecting signs of carpal instability on conventional radiographs.

Materials and methods: Two case-control datasets of hand and wrist radiographs were retrospectively acquired at three hospitals (hospitals A, B, and C). Dataset 1 (2178 radiographs from 1993 patients, hospitals A and B, 2018-2019) was used for developing an AI system for measuring scapholunate (SL) joint distances, SL and capitolunate (CL) angles, and carpal arc interruptions. Dataset 2 (481 radiographs from 217 patients, hospital C, 2017-2021) was used for testing, and with a subsample (174 radiographs from 87 patients), an observer study was conducted to compare its performance to five clinicians. Evaluation metrics included mean absolute error (MAE), sensitivity, and specificity.

Results: Dataset 2 included 258 SL distances, 189 SL angles, 191 CL angles, and 217 carpal arc labels obtained from 217 patients (mean age, 51 years ± 23 [standard deviation]; 133 women). The MAE in measuring SL distances, SL angles, and CL angles was respectively 0.65 mm (95%CI: 0.59, 0.72), 7.9 degrees (95%CI: 7.0, 8.9), and 5.9 degrees (95%CI: 5.2, 6.6). The sensitivity and specificity for detecting arc interruptions were 83% (95%CI: 74, 91) and 64% (95%CI: 56, 71). The measurements were largely comparable to those of the clinicians, while arc interruption detections were more accurate than those of most clinicians.

Conclusion: This study demonstrates that a newly developed automated AI system accurately measures and detects signs of carpal instability on conventional radiographs.

Clinical relevance statement: This system has the potential to improve detections of carpal arc interruptions and could be a promising tool for supporting clinicians in detecting carpal instability.

Keywords: Artificial intelligence; Radiography; Wrist.

Fig. 1

Flowchart for the inclusion and exclusion of studies in dataset 2 (test set). The number of studies at each step is denoted with n. Studies were preselected from the text search results in random order. ^aBased on the study metadata (e.g., study date, patient demographics, report), these studies were found to be duplicates of studies that were already included

Fig. 2

Overview of the (AI) pipeline for measuring and detecting signs of carpal instability in frontal and lateral view radiographs. The spatial and geometric properties of the relevant carpal bones are determined by segmentation and are then used to identify the articular facet joint surfaces. Based on the obtained bone surfaces and angles, the carpal instability measurements and detections can be conducted. The generated carpal arcs are visualized as color-coded points (n = 100) that form an easily interpretable heatmap. The warmer colors indicate significant deviations from the reconstructed hypothetical normal arcs (expressed as z-scores). These deviations or distances are shown by the small tails attached to the points (displacement vectors). More information can be found in Appendix E6 (online)

Fig. 3

Bland-Altman plots of the measurement agreement between the AI system and the GT on the measurements of the SL distance (n = 258, see a), SL angle (n = 189, see b), and CL angle (n = 191, see c) in dataset 2. Each marker represents one paired measurement. The dashed lines represent the mean difference (blue) and LoA (orange). The shaded bands represent 95%CI

Fig. 4

a ROC curve with the operating point of the carpal arc interruption detection results of the AI system on dataset 2 (70 positive cases, 147 negative cases). b ROC curves of the carpal arc interruption detection results of the AI system and those of the clinicians on the observer study subset (44 positive cases, 43 negative cases). Each case represents one study from one patient. The shaded bands represent 95%CIs. The black line represents no ability to discriminate between interrupted and non-interrupted arcs. AUC = area under the ROC curve

Fig. 5

Example measurement and subsequent detection errors of abnormal SL joint distances, SL angles, and CL angles made by the AI system and clinicians. The lines in yellow, cyan, and orange, respectively, represent the AI, clinicians, and GT measurements. The axes of the angle measurements are shown in white with a dense pattern (AI or clinicians) and a dashed pattern (GT). The start and end coordinates of the lines corresponding to clinicians and GT have been averaged for this figure. The measurement value and corresponding GT are provided below each panel. a 77-year-old male with a widened SL joint distance (> 3 mm). b 37-year-old male with a normal SL joint distance. c 36-year-old male female with a normal SL angle. d 37-year-old male with a normal CL angle. e 74-year-old female with a normal SL angle. f 68-year-old female with an abnormal CL angle (> 30 degrees)

Fig. 6

Example detection errors of carpal arc interruptions made by the AI system (a) and clinicians (b and c). The AI prediction and GT label are shown in the upper left corner of each image. The interruption scores of the clinicians are shown below each image (ranging from 0% [no interruption] to 100% [interruption]). The carpal arcs generated by the AI system are overlaid as color-coded points on the original image. The points correspond to z-scores: the higher the z-score, the more abnormal and hence indicative the point is of an interruption (see more information in Appendix E6 [online]). The deviations from the hypothetical normal shape of the carpal arcs are shown by the small tails attached to the points (displacement vectors). a 53-year-old female with a distal radius fracture and slight narrowing of the radiocarpal joint but normal carpal alignment. b 82-year-old male with the capitate subluxating proximally into the direction of a widened SL joint space. c 26-year-old male with slight angulation of the lunate accompanied by a widened SL joint space and semiacute scaphoid fracture. ER Doc = emergency doctor, H Surg = hand surgeon, Jr Doc = junior doctor, MSK Rad = musculoskeletal radiologist, Rad = radiologist