Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jun;26(3):745-749.
doi: 10.1007/s10029-021-02488-4. Epub 2021 Aug 22.

Randomized controlled trials published on patients with hernias have a high percentage of unreproducible statistics

Naila H Dhanani  1 , Oscar A Olavarria  2 , Cynthia S Bell  3 , Julie L Holihan  2 , Mike K Liang  4
Affiliations

Randomized controlled trials published on patients with hernias have a high percentage of unreproducible statistics

Naila H Dhanani et al. Hernia. 2022 Jun.

Erratum in

Abstract

Introduction: Surgeons rely on randomized controlled trials (RCT) to compare the effectiveness of treatments. RCTs require careful planning and substantial effort to complete. Because of the careful study design, statistics performed are often easy to reproduce such as Chi-squared or t-test. Issues such as statistical discordance, or reporting statistical results that cannot be reproduced, should be uncommon.

Methods: RCTs pertaining to hernias were identified in PubMed using the search terms "hernia" and "randomized controlled trial." Studies were selected using a random number generator. Studies were included if the primary outcome could be reproduced using the data and statistical test reported in the manuscript. Discordance between the obtained p-value from our analysis and the published p-value was assessed. Primary outcome was the number of studies that reported p-values that crossed the level of statistical significance (p-value = 0.05) but on reproduction analysis did not.

Results: Of the 100 included RCTs, five reported p-values that crossed the "p = 0.05" threshold that our team was unable to reproduce using the statistical test reported in the manuscript. An additional three studies reported p-values that crossed the "p = 0.05" threshold that our team was unable to reproduce using the appropriate statistical test (i.e., Fisher's exact test when all expected cell counts < 5). All eight studies published p-values < 0.05, whereas, our re-analysis demonstrated p ≥ 0.05.

Conclusion: Eight percent of the RCTs analyzed in this study reported p-values < 0.05 that on reproduction analysis was ≥ 0.05. The next steps should be to determine reasons for discordance and how to prevent this from happening.

Keywords: Discordance; Hernia; Randomized controlled trials; Statistics; p-value.

PubMed Disclaimer

Comment in

References

    1. GBD (2015) Disease and Injury Incidence and Prevalence Collaborators (2016) Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 388(10053):1545–1602. https://doi.org/10.1016/S0140-6736(16)31678-6 (Erratum.In:Lancet.2017Jan7;389(10064):e1) - DOI
    1. JT Sarosi GA Jr, Syme WC, Thompson JS, Wang J, Jonasson O, (2006) Watchful waiting vs repair of inguinal hernia in minimally symptomatic men: a randomized clinical trial. JAMA 295(3):285–292. https://doi.org/10.1001/jama.295.3.285 (Erratum.In:JAMA.2006Jun21;295(23):2726) - DOI
    1. Fitzgibbons RJ Jr, Ramanan B, Arya S et al (2013) Long-term results of a randomized controlled trial of a nonoperative strategy (watchful waiting) for men with minimally symptomatic inguinal hernias. Ann Surg 258(3):508–515. https://doi.org/10.1097/SLA.0b013e3182a19725 - DOI - PubMed
    1. de Goede B, Wijsmuller AR, van Ramshorst GH et al (2018) Watchful waiting versus surgery of mildly symptomatic or asymptomatic inguinal hernia in men aged 50 years and older: a randomized controlled trial. Ann Surg 267(1):42–49. https://doi.org/10.1097/SLA.0000000000002243 - DOI - PubMed
    1. Kaufmann R, Halm JA, Eker HH et al (2018) Mesh versus suture repair of umbilical hernia in adults: a randomised, double-blind, controlled, multicentre trial. Lancet 391(10123):860–869. https://doi.org/10.1016/S0140-6736(18)30298-8 - DOI - PubMed

LinkOut - more resources