The Continuous Fragility Index of Statistically Significant Findings in Randomized Controlled Trials That Compare Interventions for Anterior Shoulder Instability

Abstract

Background: Evidence-based care relies on robust research. The fragility index (FI) is used to assess the robustness of statistically significant findings in randomized controlled trials (RCTs). While the traditional FI is limited to dichotomous outcomes, a novel tool, the continuous fragility index (CFI), allows for the assessment of the robustness of continuous outcomes.

Purpose: To calculate the CFI of statistically significant continuous outcomes in RCTs evaluating interventions for managing anterior shoulder instability (ASI).

Study design: Meta-analysis; Level of evidence, 2.

Methods: A search was conducted across the MEDLINE, Embase, and CENTRAL databases for RCTs assessing management strategies for ASI from inception to October 6, 2022. Studies that reported a statistically significant difference between study groups in ≥1 continuous outcome were included. The CFI was calculated and applied to all available RCTs reporting interventions for ASI. Multivariable linear regression was performed between the CFI and various study characteristics as predictors.

Results: There were 27 RCTs, with a total of 1846 shoulders, included. The median sample size was 61 shoulders (IQR, 43). The median CFI across 27 RCTs was 8.2 (IQR, 17.2; 95% CI, 3.6-15.4). The median CFI was 7.9 (IQR, 21; 95% CI, 1-22) for 11 studies comparing surgical methods, 22.6 (IQR, 16; 95% CI, 8.2-30.4) for 6 studies comparing nonsurgical reduction interventions, 2.8 for 3 studies comparing immobilization methods, and 2.4 for 3 studies comparing surgical versus nonsurgical interventions. Significantly, 22 of 57 included outcomes (38.6%) from studies with completed follow-up data had a loss to follow-up exceeding their CFI. Multivariable regression demonstrated that there was a statistically significant positive correlation between a trial's sample size and the CFI of its outcomes (r = 0.23 [95% CI, 0.13-0.33]; P < .001).

Conclusion: More than a third of continuous outcomes in ASI trials had a CFI less than the reported loss to follow-up. This carries the significant risk of reversing trial findings and should be considered when evaluating available RCT data. We recommend including the FI, CFI, and loss to follow-up in the abstracts of future RCTs.

Keywords: continuous fragility index; fragility index; shoulder general; shoulder instability; statistical significance; statistics.

Figure 1.

Calculating the continuous fragility index (CFI). The outcomes for 2 theoretical patient groups are illustrated, with each outcome represented as a diamond on the number line. (A) Before any manipulation, there is a statistically significant difference between the groups (P < .0001). We first identify the data points closest to and greater than the mean in the intervention group. These data points are moved 1 unit at a time in a sequential manner to the control group until the 2 groups are no longer statistically different (resulting in B). In this case, 5 diamonds (C) must be moved. Therefore, the robustness of the initial difference can be described with a CFI value of 5.

Figure 2.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart.

Figure 3.

Histogram of study-specific continuous fragility index (CFI) values.