Inter-Rater Reliability of the Modified Radiographic Union Score for Diaphyseal Tibial Fractures With Bone Defects

Abstract

Objectives: To evaluate inter-rater reliability of the modified Radiographic Union Score for Tibial (mRUST) fractures among patients with open, diaphyseal tibia fractures with a bone defect treated with intramedullary nails (IMNs), plates, or definitive external fixation (ex-fix).

Design: Retrospective cohort study.

Setting: Fifteen-level one civilian trauma centers; 2 military treatment facilities.

Patients/participants: Patients ≥18 years old with open, diaphyseal tibia fractures with a bone defect ≥1 cm surgically treated between 2007 and 2012.

Intervention: Three of 6 orthopedic traumatologists reviewed and applied mRUST scoring criteria to radiographs from the last clinical visit within 13 months of injury.

Main outcome measurements: Inter-rater reliability was assessed using Krippendorff's alpha (KA) statistic; intraclass correlation coefficient (ICC) is presented for comparison with previous publications.

Results: Two hundred thirteen patients met inclusion criteria including 115 IMNs, 24 plates, 29 ex-fixes, and 45 cases that no longer had instrumentation at evaluation. All reviewers agreed on the pattern of scoreable cortices for 90.4% of IMNs, 88.9% of those without instrumentation, 44.8% of rings, and 20.8% of plates. Thirty-one (15%) cases, primarily plates and ex-fixes, did not contribute to KA and ICC estimates because <2 raters scored all cortices. The overall KA for the 85% that could be analyzed was 0.64 (ICC 0.71). For IMNs, plates, ex-fixes, and no instrumentation, KA (ICC) was 0.65 (0.75), 0.88 (0.90), 0.47 (0.62), and 0.48 (0.57), respectively.

Conclusions: In tibia fractures with bone defects, the mRUST seems similarly reliable to previous work in patients treated with IMN but is less reliable in those with plates or ex-fixes, or after removal of instrumentation.

Figure 1.. Flow Diagram of Patient Selection

Figure 2.. Visualizing disagreement in mRUST by type, or lack of, instrumentation visible on final radiograph applied to 213 cases.

Each rater is represented by a symbol with “/”, “X”, and “⊗” representing agreement among the three raters. Presence of the “X” symbol indicates that two (of three) raters were in perfect agreement for a given case, and presence of the “⊗” symbol indicates that all three raters were in perfect agreement. The “/” symbol indicates that a single rater provided a different assessment than the other raters. For included cases, the vertical line segment connects the minimum and maximum scores for a single patient. The figure is stratified into four panels by instrumentation group. Within each panel, scoring disagreement (as measured by the length of the line segments) increases from left to right; non-contributing cases (≥2 N/As) are placed furthest to the right. Multiple cases straddle a previously defined score indicating union (mRUST=13 (blue line)). Thirty-nine (18%) cases had ≥1 score above and another score below the threshold (IMN: 15%, None: 38%, Plate: 0%, Ring: 17%). “IMN” indicates intramedullary nail; “None” indicates cases with no instrumentation seen on rated radiographs; “N/A” indicates that a score could not be computed; Alpha indicates Krippendorff’s Alpha; ICC indicates Intraclass Correlation Coefficient

Figure 3.. Radiographs from a patient with a clinically healed tibia fracture showing persistent abnormal appearance of the bone

This is an example of a case that may be unreliably scored under the mRUST protocol.