P-Hacking in Orthopaedic Literature: A Twist to the Tail

Abstract

Background: "P-hacking" occurs when researchers preferentially select data or statistical analyses until nonsignificant results become significant. We wanted to evaluate if the phenomenon of p-hacking was evident in orthopaedic literature.

Methods: We text-mined through all articles published in three top orthopaedic journals in 2015. For anonymity, we cipher-coded the three journals. We included all studies that reported a single p value to answer their main hypothesis. These p values were then charted and frequency graphs were generated to illustrate any evidence of p-hacking. Binomial tests were employed to look for evidence of evidential value and significance of p-hacking.

Results: Frequency plots for all three journals revealed evidence of p-hacking. Binomial tests for all three journals were significant for evidence of evidential value (p < 0.0001 for all). However, the binomial test for p-hacking was significant only for one journal (p = 0.0092).

Conclusions: P-hacking is an evolving phenomenon that threatens to jeopardize the evidence-based practice of medicine. Although our results show that there is good evidential value for orthopaedic literature published in our top journals, there is some evidence of p-hacking of which authors and readers should be wary.