Artificial intelligence-based detection of paediatric appendicular skeletal fractures: performance and limitations for common fracture types and locations

Abstract

Background: Research into artificial intelligence (AI)-based fracture detection in children is scarce and has disregarded the detection of indirect fracture signs and dislocations.

Objective: To assess the diagnostic accuracy of an existing AI-tool for the detection of fractures, indirect fracture signs, and dislocations.

Materials and methods: An AI software, BoneView (Gleamer, Paris, France), was assessed for diagnostic accuracy of fracture detection using paediatric radiology consensus diagnoses as reference. Radiographs from a single emergency department were enrolled retrospectively going back from December 2021, limited to 1,000 radiographs per body part. Enrolment criteria were as follows: suspected fractures of the forearm, lower leg, or elbow; age 0-18 years; and radiographs in at least two projections.

Results: Lower leg radiographs showed 607 fractures. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were high (87.5%, 87.5%, 98.3%, 98.3%, respectively). Detection rate was low for toddler's fractures, trampoline fractures, and proximal tibial Salter-Harris-II fractures. Forearm radiographs showed 1,137 fractures. Sensitivity, specificity, PPV, and NPV were high (92.9%, 98.1%, 98.4%, 91.7%, respectively). Radial and ulnar bowing fractures were not reliably detected (one out of 11 radial bowing fractures and zero out of seven ulnar bowing fractures were correctly detected). Detection rate was low for styloid process avulsions, proximal radial buckle, and complete olecranon fractures. Elbow radiographs showed 517 fractures. Sensitivity and NPV were moderate (80.5%, 84.7%, respectively). Specificity and PPV were high (94.9%, 93.3%, respectively). For joint effusion, sensitivity, specificity, PPV, and NPV were moderate (85.1%, 85.7%, 89.5%, 80%, respectively). For elbow dislocations, sensitivity and PPV were low (65.8%, 50%, respectively). Specificity and NPV were high (97.7%, 98.8%, respectively).

Conclusions: The diagnostic performance of BoneView is promising for forearm and lower leg fractures. However, improvement is mandatory before clinicians can rely solely on AI-based paediatric fracture detection using this software.

Keywords: Appendicular skeleton; Artificial intelligence; Fracture; Paediatric; Radiograph.

Fig. 1

Examples of lower leg radiographs showing true positive (a–d), false positive (e–h) and false negative (i–k) diagnoses made by BoneView (Gleamer, Paris, France). a–d Anteroposterior (AP) and lateral radiographs of the left lower leg in a 5-year-old girl before (a,b) and after (c,d) annotation by BoneView. There is a tibial spiral fracture. The fracture was detected with a confidence score above 90% (rectangles/“FRACT”). e–h AP and lateral radiographs of the left lower leg in an 11-month-old girl before (e,f) and after (g,h) annotation by BoneView. There is a distal tibial Salter-Harris-II fracture. The true positive finding of a fracture was detected with a confidence score above 90% (rectangles/“FRACT”). Additionally, BoneView detected a false positive proximal fracture on the lateral radiograph (h) with a confidence score between 50% and 90% (broken rectangle/“FRACT”). i–k AP and lateral radiographs of the left lower leg in a 1-year-old boy before (i,j) and after (k) annotation by BoneView. There is a proximal metaphyseal tibial buckle fracture (trampoline fracture), which is only visible on the lateral radiograph (rectangle). BoneView did not detect the fracture

Fig. 2

Examples of forearm radiographs showing true positive (a–d), false positive (e–h) and false negative (i–k) diagnoses made by BoneView (Gleamer, Paris, France). a–d Anteroposterior (AP) and lateral radiographs of the left forearm in a 5-year-old boy before (a,b) and after (c,d) annotation by BoneView. There are distal metaphyseal radial and ulnar buckle fractures. The fractures were detected with a confidence score above 90% (rectangles/“FRACT”). e–h AP and lateral radiographs of the left forearm in a 5-year-old boy before (e,f) and after (g,h) annotation by BoneView. The radiographs were normal. BoneView detected a false positive distal diaphyseal ulnar fracture on the lateral radiograph with a confidence score above 90% (rectangle/“FRACT”). Additionally, false positive effusion in the elbow joint was detected (rectangle/“EFF”). i–k AP and lateral radiographs of the left forearm in a 2-year-old boy before (i,j) and after (k) annotation by BoneView. There is a radial bowing fracture which is only visible on the lateral radiograph (rectangle). BoneView did not detect the fracture

Fig. 3

Examples of elbow radiographs showing true positive (a–d), false positive (e–h) and false negative (i–k) diagnoses made by BoneView (Gleamer, Paris, France). a–d Anteroposterior (AP) and lateral radiographs of the left elbow in a 4-year-old girl before (a,b) and after (c,d) annotation by BoneView. There is a complete supracondylar fracture with effusion (positive anterior and posterior fat pad sign). The fracture was detected with a confidence score above 90% (rectangles/“FRACT”). The effusion was depicted correctly by BoneView (rectangle/“EFF”). e–h AP and lateral radiographs of the right elbow in a 10-year-old girl before (e,f) and after (g,h) annotation by BoneView. The radiographs were normal. BoneView detected an avulsion of the radial epicondyle with a confidence score above 90% (rectangle/“FRACT”). i–l AP and lateral radiographs of the right elbow in a 6-year-old boy before (i,j) and after (k) annotation by BoneView. There is a Monteggia fracture-dislocation. BoneView detected the true positive complete fracture of the olecranon with a confidence score above 90% (orange rectangle/“FRACT”). The associated proximal radial dislocation was disregarded (red rectangle)