Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comment
. 2023 Jan 17;120(3):e2218957120.
doi: 10.1073/pnas.2218957120. Epub 2023 Jan 9.

Variation across analysts in statistical significance, yet consistently small effect sizes

Maya B Mathur  1 , Christian Covington  2 , Tyler J VanderWeele  3
Affiliations
Comment

Variation across analysts in statistical significance, yet consistently small effect sizes

Maya B Mathur et al. Proc Natl Acad Sci U S A. .
No abstract available

PubMed Disclaimer

Conflict of interest statement

The authors declare competing interests. The authors have organizational affiliations to disclose, M.B.M. is Associate Director of the Stanford University Center for Open and Reproducible Science and is a member of the research advisory boards of Sentience Institute and Greener By Default. T.J.V. has received personal fees from Flerish and from Flourishing Metrics.

Comment on

  • Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty.
    Breznau N, Rinke EM, Wuttke A, Nguyen HHV, Adem M, Adriaans J, Alvarez-Benjumea A, Andersen HK, Auer D, Azevedo F, Bahnsen O, Balzer D, Bauer G, Bauer PC, Baumann M, Baute S, Benoit V, Bernauer J, Berning C, Berthold A, Bethke FS, Biegert T, Blinzler K, Blumenberg JN, Bobzien L, Bohman A, Bol T, Bostic A, Brzozowska Z, Burgdorf K, Burger K, Busch KB, Carlos-Castillo J, Chan N, Christmann P, Connelly R, Czymara CS, Damian E, Ecker A, Edelmann A, Eger MA, Ellerbrock S, Forke A, Forster A, Gaasendam C, Gavras K, Gayle V, Gessler T, Gnambs T, Godefroidt A, Grömping M, Groß M, Gruber S, Gummer T, Hadjar A, Heisig JP, Hellmeier S, Heyne S, Hirsch M, Hjerm M, Hochman O, Hövermann A, Hunger S, Hunkler C, Huth N, Ignácz ZS, Jacobs L, Jacobsen J, Jaeger B, Jungkunz S, Jungmann N, Kauff M, Kleinert M, Klinger J, Kolb JP, Kołczyńska M, Kuk J, Kunißen K, Kurti Sinatra D, Langenkamp A, Lersch PM, Löbel LM, Lutscher P, Mader M, Madia JE, Malancu N, Maldonado L, Marahrens H, Martin N, Martinez P, Mayerl J, Mayorga OJ, McManus P, McWagner K, Meeusen C, Meierrieks D, Mellon J, Merhout F, Merk S, Meyer D, Micheli L, Mijs J, Moya C, Neunhoeffer M, Nüst D, Nygård O, Ochsenfeld F, Otte G, Pechenkina AO… See abstract for full author list ➔ Breznau N, et al. Proc Natl Acad Sci U S A. 2022 Nov;119(44):e2203150119. doi: 10.1073/pnas.2203150119. Epub 2022 Oct 28. Proc Natl Acad Sci U S A. 2022. PMID: 36306328 Free PMC article.

References

    1. Breznau N., et al. , Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proc. Natl. Acad. Sci. U.S.A. 119, e2203150119 (2022). - PMC - PubMed
    1. Cummings P., Arguments for and against standardized mean differences (effect sizes). Arch. Pediatr. Adolesc. Med. 165, 592–596 (2011). - PubMed
    1. Cohen J., Statistical Power Analysis for the Behavioral Sciences (Routledge, 2013).
    1. Mathur M., VanderWeele T., Robust metrics and sensitivity analyses for meta-analyses of heterogeneous effects. Epidemiology 31, 356–358 (2020). - PMC - PubMed
    1. Wang C., Lee W.-C., A simple method to estimate prediction intervals and predictive distributions: Summarizing meta-analyses beyond means and confidence intervals. Res. Synth. Methods 10, 255–266 (2019). - PubMed

LinkOut - more resources