Open Science Standards at Journals that Inform Evidence-Based Policy

doi:10.1007/s11121-023-01543-z

Review

. 2023 Oct;24(7):1275-1291.

doi: 10.1007/s11121-023-01543-z. Epub 2023 May 13.

Open Science Standards at Journals that Inform Evidence-Based Policy

Sean Grant^{1

2}, Evan Mayo-Wilson^{3

4}, Sina Kianersi^{4

5}, Kevin Naaman^{4

6}, Beate Henschel⁴

Affiliations

¹ HEDCO Institute for Evidence-Based Educational Practice, College of Education, University of Oregon, OR, 97403-1215, Eugene, USA. spgrant@uoregon.edu.
² Richard M. Fairbanks School of Public Health, Indiana University, Indianapolis, IN, USA. spgrant@uoregon.edu.
³ Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC, USA.
⁴ School of Public Health-Bloomington, Indiana University, Bloomington, IN, USA.
⁵ Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA.
⁶ Indiana University, School of Education, Bloomington, IN, USA.

PMID: 37178346
DOI: 10.1007/s11121-023-01543-z

Review

Open Science Standards at Journals that Inform Evidence-Based Policy

Sean Grant et al. Prev Sci. 2023 Oct.

. 2023 Oct;24(7):1275-1291.

doi: 10.1007/s11121-023-01543-z. Epub 2023 May 13.

Authors

Sean Grant^{1

2}, Evan Mayo-Wilson^{3

4}, Sina Kianersi^{4

5}, Kevin Naaman^{4

6}, Beate Henschel⁴

Affiliations

¹ HEDCO Institute for Evidence-Based Educational Practice, College of Education, University of Oregon, OR, 97403-1215, Eugene, USA. spgrant@uoregon.edu.
² Richard M. Fairbanks School of Public Health, Indiana University, Indianapolis, IN, USA. spgrant@uoregon.edu.
³ Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC, USA.
⁴ School of Public Health-Bloomington, Indiana University, Bloomington, IN, USA.
⁵ Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA.
⁶ Indiana University, School of Education, Bloomington, IN, USA.

PMID: 37178346
DOI: 10.1007/s11121-023-01543-z

Abstract

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Keywords: Evidence clearinghouse; Evidence-based policy; Open science; TOP guidelines.

PubMed Disclaimer

Cited by

International initiatives to enhance awareness and uptake of open research in psychology: a systematic mapping review.
Skubera M, Korbmacher M, Evans TR, Azevedo F, Pennington CR. Skubera M, et al. R Soc Open Sci. 2025 Mar 19;12(3):241726. doi: 10.1098/rsos.241726. eCollection 2025 Mar. R Soc Open Sci. 2025. PMID: 40109933 Free PMC article. Review.
A guide for social science journal editors on easing into open science.
Silverstein P, Elman C, Montoya A, McGillivray B, Pennington CR, Harrison CH, Steltenpohl CN, Röer JP, Corker KS, Charron LM, Elsherif M, Malicki M, Hayes-Harb R, Grinschgl S, Neal T, Evans TR, Karhulahti VM, Krenzer WLD, Belaus A, Moreau D, Burin DI, Chin E, Plomp E, Mayo-Wilson E, Lyle J, Adler JM, Bottesini JG, Lawson KM, Schmidt K, Reneau K, Vilhuber L, Waltman L, Gernsbacher MA, Plonski PE, Ghai S, Grant S, Christian TM, Ngiam W, Syed M. Silverstein P, et al. Res Integr Peer Rev. 2024 Feb 16;9(1):2. doi: 10.1186/s41073-023-00141-5. Res Integr Peer Rev. 2024. PMID: 38360805 Free PMC article.
Investigation of research quality and transparency in neurosurgery through the utilization of open science practices.
Alam Z, Desai K, Maddali A, Sivan V, Kumar RP, O'Malley GR, Patel N. Alam Z, et al. Neurosurg Rev. 2024 Oct 8;47(1):750. doi: 10.1007/s10143-024-03008-5. Neurosurg Rev. 2024. PMID: 39377867

References

1. Altman, D. G., Furberg, C. D., Grimshaw, J. M., & Shanahan, D. R. (2014). Linked publications from a single trial: A thread of evidence. Trials, 15(1), 369. https://doi.org/10.1186/1745-6215-15-369 - DOI - PubMed - PMC
1. Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022). Promoting learning from null or negative results in prevention science trials. Prevention Science, 23(5), 751–763. - PubMed - DOI
1. Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), Article 7604.
1. Bakker, B. N., Jaidka, K., Dörr, T., Fasching, N., & Lelkes, Y. (2021). Questionable and open research practices: Attitudes and perceptions among quantitative communication researchers. Journal of Communication, 71(5), 715–738. - DOI
1. Baron, J. (2018). A brief history of evidence-based policy. The ANNALS of the American Academy of Political and Social Science, 678(1), 40–50. - DOI

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
- Springer

[1] Altman, D. G., Furberg, C. D., Grimshaw, J. M., & Shanahan, D. R. (2014). Linked publications from a single trial: A thread of evidence. Trials, 15(1), 369. https://doi.org/10.1186/1745-6215-15-369 - DOI - PubMed - PMC

[2] Altman, D. G., Furberg, C. D., Grimshaw, J. M., & Shanahan, D. R. (2014). Linked publications from a single trial: A thread of evidence. Trials, 15(1), 369. https://doi.org/10.1186/1745-6215-15-369 - DOI - PubMed - PMC

[3] Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022). Promoting learning from null or negative results in prevention science trials. Prevention Science, 23(5), 751–763. - PubMed - DOI

[4] Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022). Promoting learning from null or negative results in prevention science trials. Prevention Science, 23(5), 751–763. - PubMed - DOI

[5] Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), Article 7604.

[6] Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), Article 7604.

[7] Bakker, B. N., Jaidka, K., Dörr, T., Fasching, N., & Lelkes, Y. (2021). Questionable and open research practices: Attitudes and perceptions among quantitative communication researchers. Journal of Communication, 71(5), 715–738. - DOI

[8] Bakker, B. N., Jaidka, K., Dörr, T., Fasching, N., & Lelkes, Y. (2021). Questionable and open research practices: Attitudes and perceptions among quantitative communication researchers. Journal of Communication, 71(5), 715–738. - DOI

[9] Baron, J. (2018). A brief history of evidence-based policy. The ANNALS of the American Academy of Political and Social Science, 678(1), 40–50. - DOI

[10] Baron, J. (2018). A brief history of evidence-based policy. The ANNALS of the American Academy of Political and Social Science, 678(1), 40–50. - DOI

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Open Science Standards at Journals that Inform Evidence-Based Policy

Affiliations

Open Science Standards at Journals that Inform Evidence-Based Policy

Authors

Affiliations

Abstract

Similar articles

Cited by

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources