The fragility of significant results underscores the need of larger randomized controlled trials in nephrology

Abstract

The Fragility Index is a tool for testing robustness of randomized controlled trial results for dichotomous outcomes. It describes the minimum number of individuals in whom changing an event status would render a statistically significant result nonsignificant. Here we identified all randomized controlled trials in five nephrology and five general journals from 2005-2014. A total of 127 randomized controlled trials reporting at least one dichotomous statistically significant outcome (p less than 0.05) were included and the Fragility Index was calculated. Twenty randomized controlled trials had a Fragility Index of zero and were excluded from further analysis. Linear regression was performed to assess factors associated with Fragility Indexes stratified by primary or secondary outcomes. The median sample size was 134 (range 2211506) with 36 (range 5-2743) total number of events. The median Fragility Index was three (range 1-166), indicating that in half the trials the addition of three events to the treatment with the lowest number of events rendered the result nonsignificant. For primary outcome studies a doubling in total event number and sample size significantly increased the geometric mean Fragility Index by 52% and 42%, respectively. Compared to a reported p value of 0.05 to 0.01, those reporting 0.01 to 0.001 or less than 0.001 had a significant 57% and 472% increase in the median Fragility Index, respectively. Forty-one percent had a Fragility Index less than the total loss to follow-up, indicating a potential to change a trial result had all individuals been accounted for. Thus, our study highlights the need for larger randomized controlled trials with accurate accounting for loss to follow-up to adequately guide evidence-based practice.

Keywords: fragility score; nephrology; randomized controlled trials.