A Prospective Approach to Integration of AI Fracture Detection Software in Radiographs into Clinical Workflow

Abstract

Gleamer BoneView^© is a commercially available AI algorithm for fracture detection in radiographs. We aim to test if the algorithm can assist in better sensitivity and specificity for fracture detection by residents with prospective integration into clinical workflow. Radiographs with inquiry for fracture initially reviewed by two residents were randomly assigned and included. A preliminary diagnosis of a possible fracture was made. Thereafter, the AI decision on presence and location of possible fractures was shown and changes to diagnosis could be made. Final diagnosis of fracture was made by a board-certified radiologist with over eight years of experience, or if available, cross-sectional imaging. Sensitivity and specificity of the human report, AI diagnosis, and assisted report were calculated in comparison to the final expert diagnosis. 1163 exams in 735 patients were included, with a total of 367 fractures (31.56%). Pure human sensitivity was 84.74%, and AI sensitivity was 86.92%. Thirty-five changes were made after showing AI results, 33 of which resulted in the correct diagnosis, resulting in 25 additionally found fractures. This resulted in a sensitivity of 91.28% for the assisted report. Specificity was 97.11, 84.67, and 97.36%, respectively. AI assistance showed an increase in sensitivity for both residents, without a loss of specificity.

Keywords: artificial intelligence; computer-aided diagnosis; fracture; radiographs.

Figure 1

Result frames shown by the AI software delineating a case of exams without a fracture ((a), left), a possible fracture at 50–89% certainty threshold ((b), center) and a fracture at ≥90% certainty ((c), right). The number X-ray images analyzed in the exam is also shown.

Figure 2

Clinical workflow for fracture diagnosis without (white boxes) and with (gray boxes) AI assistance.

Figure 3

(a) shows a lateral radiograph of the left ankle. The bounding box (white) marks a fracture noted by the AI software. This was initially thought to be an osteophyte by both the resident and the board-certified radiologist. (b) Sagittal CT imaging confirms a true positive fracture of the Navicular (white arrow).

Figure 4

Standing ap radiograph of the left shoulder. The white arrow marks an incidentally caught rib fracture, which was correctly diagnosed by the resident but missed by the AI software.

Figure 5

Standing ap radiograph of the right shoulder. The AI software correctly sets a bounding box around the acromioclavicular joint injury (marked as “Dislocation”); however, both the AI and the resident missed the displaced proximal clavicle fracture (white arrow).

Figure 6

Lateral radiograph of the lumbar spine. Multiple (older) vertebral compression fractures are correctly marked as “Positive” (through line bounding box) and “Doubt” (dashed line bounding box). The bottom bounding box marks the intervertebral space L4/5 (arrow) an obvious mistake of the AI.

Figure 7

Lateral radiograph of the lumbar spine. In multiple such images, the AI incorrectly marks rib overlay as a possible fracture, noted here by the dashed bounding box marking a “Doubt” fracture.