The statistical fragility of studies on rotator cuff repair with graft augmentation

Abstract

Background: Clinical decision-making often relies on evidence-based medicine. Our purpose was to determine the fragility index (FI) and fragility quotient (FQ) for studies evaluating rotator cuff repair (RCR) with graft augmentation. A lost to follow-up (LTF) value greater than the FI indicates statistical instability for the reported outcomes and conclusions.

Methods: We performed a systematic review using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines by searching PubMed, the Cochrane library, and Embase in June 2022 to identify studies of RCR with graft augmentation. Comparative studies with at least 1 statistically analyzed dichotomous outcome were included. Seventeen studies published in seven peer-reviewed journals from 2003 to 2019 were subsequently evaluated. The FI was determined by changing each reported outcome event within 2 × 2 contingency tables until statistical significance or nonsignificance was reversed. The associated FQ was determined by dividing the FI by the sample size. LTF values were also extracted from each included study.

Results: The included studies had a total of 1098 patients with 36 dichotomous outcomes. The associated median FI was 4 (interquartile range 2-5), indicating that the reversal of 4 patients' outcomes would have reversed the finding of significant difference. The median FQ was 0.08 (interquartile range 0.04-0.15), indicating that in a sample of 100 patients, reversal of 8 patients' outcomes would reverse statistical significance. The median number of patients LTF was 3 (range 0-25), with 56% of reported outcomes having LTF greater than their respective FI.

Conclusion: Studies of RCR with graft augmentation lack statistical stability, with few altered outcome events required to reverse statistical significance. Larger comparative studies with better follow-up will strengthen the statistical stability of the evidence for RCR with graft augmentation. For future investigations and reports, we recommend including FI and FQ along with traditional statistical significance analyses to provide better context on the strength of conclusions.

Keywords: Fragility index; augmentation; fragility quotient; graft; rotator cuff.