The Feature Ambiguity Mitigate Operator model helps improve bone fracture detection on X-ray radiograph

Abstract

This study was performed to propose a method, the Feature Ambiguity Mitigate Operator (FAMO) model, to mitigate feature ambiguity in bone fracture detection on radiographs of various body parts. A total of 9040 radiographic studies were extracted. These images were classified into several body part types including 1651 hand, 1302 wrist, 406 elbow, 696 shoulder, 1580 pelvic, 948 knee, 1180 ankle, and 1277 foot images. Instance segmentation was annotated by radiologists. The ResNext-101+FPN was employed as the baseline network structure and the FAMO model for processing. The proposed FAMO model and other ablative models were tested on a test set of 20% total radiographs in a balanced body part distribution. To the per-fracture extent, an AP (average precision) analysis was performed. For per-image and per-case, the sensitivity, specificity, and AUC (area under the receiver operating characteristic curve) were analyzed. At the per-fracture level, the controlled experiment set the baseline AP to 76.8% (95% CI: 76.1%, 77.4%), and the major experiment using FAMO as a preprocessor improved the AP to 77.4% (95% CI: 76.6%, 78.2%). At the per-image level, the sensitivity, specificity, and AUC were 61.9% (95% CI: 58.7%, 65.0%), 91.5% (95% CI: 89.5%, 93.3%), and 74.9% (95% CI: 74.1%, 75.7%), respectively, for the controlled experiment, and 64.5% (95% CI: 61.3%, 67.5%), 92.9% (95% CI: 91.0%, 94.5%), and 77.5% (95% CI: 76.5%, 78.5%), respectively, for the experiment with FAMO. At the per-case level, the sensitivity, specificity, and AUC were 74.9% (95% CI: 70.6%, 78.7%), 91.7%% (95% CI: 88.8%, 93.9%), and 85.7% (95% CI: 84.8%, 86.5%), respectively, for the controlled experiment, and 77.5% (95% CI: 73.3%, 81.1%), 93.4% (95% CI: 90.7%, 95.4%), and 86.5% (95% CI: 85.6%, 87.4%), respectively, for the experiment with FAMO. In conclusion, in bone fracture detection, FAMO is an effective preprocessor to enhance model performance by mitigating feature ambiguity in the network.

Figure 1

Per-image level ROC curve.

Figure 2

Per-case level ROC curve.

Figure 3

Effects of the FAMO (Feature Ambiguity Mitigate Operator) plus ResNeXt101+FPN model and the ResNeXt101+FPN model only, with the FAMO plus ResNeXt101+FPN model being helpful in finding bone fractures. (A) A minor fracture was marked by the radiologist (small box) on the radiograph. (B) The fracture line was successfully recognized by the FAMO plus ResNeXt101+FPN model (green box). (C) The ResNeXt101+FPN model did not recognize the fracture. No fracture line was detected on the radiograph by the radiologist (D) or by the FAMO plus ResNeXt101+FPN model (E). But the ResNeXt101+FPN model (causing feature distortion) detected a false positive fracture line (F, green box).

Figure 4

Effects of the Feature Ambiguity Mitigate Operator (FAMO) plus ResNeXt101+FPN model and the ResNeXt101+FPN model only, with the FAMO plus ResNeXt101+FPN model being helpful in finding bone fractures. (A–C) Dislocation of the left shoulder joint. (A) The radiologist correctly located the dislocation of the shoulder joint. (B) The FAMO plus ResNeXt101+FPN model also correctly pointed out the location of dislocation. (C) The ResNeXt101+FPN model only did not find the dislocation. (D–F) Compression fracture of the calcaneus. (D) The radiologist correctly found the fracture site. (E) The FAMO plus ResNeXt101+FPN model also correctly pointed out the fracture area. (F) The ResNeXt101+FPN model failed to find the fracture. (G–I) Radial embedded fracture. (G) The radiologist correctly found the embedded fracture. (H) The FAMO plus ResNeXt101+FPN model also correctly pointed out the fracture. (I) The ResNeXt101+FPN model failed to find the fracture.

Figure 5

Object detection architecture on fractures dataset.

Figure 6

Feature ambiguity in fracture detection. When the fracture line was lying horizontally (upper left), a narrow rectangle box was annotated, and when the fracture line was in an oblique direction, a broader box was annotated (lower left). Both boxes were fed into the RoI-Align operator and resized to the same square shape (right). Processing with the RoI-Align operator caused ambiguity of the image and downgrade the model performance.

Figure 7

Feature ambiguity in fracture detection was countered in the Feature Ambiguity Mitigate Operator (FAMO) method. On the left, the annotated box was adjusted by expanding its short side (upper left) to the same length as its long side and making a square box (lower left). Thus, the image was not distorted nor ambiguous.