Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Jan 4:11:69.
doi: 10.3389/fninf.2017.00069. eCollection 2017.

Re-run, Repeat, Reproduce, Reuse, Replicate: Transforming Code into Scientific Contributions

Fabien C Y Benureau  1   2   3 , Nicolas P Rougier  1   2   3
Affiliations

Re-run, Repeat, Reproduce, Reuse, Replicate: Transforming Code into Scientific Contributions

Fabien C Y Benureau et al. Front Neuroinform. .

Abstract

Scientific code is different from production software. Scientific code, by producing results that are then analyzed and interpreted, participates in the elaboration of scientific conclusions. This imposes specific constraints on the code that are often overlooked in practice. We articulate, with a small example, five characteristics that a scientific code in computational science should possess: re-runnable, repeatable, reproducible, reusable, and replicable. The code should be executable (re-runnable) and produce the same result more than once (repeatable); it should allow an investigator to reobtain the published results (reproducible) while being easy to use, understand and modify (reusable), and it should act as an available reference for any ambiguity in the algorithmic descriptions of the article (replicable).

Keywords: best practices; computational science; replicability; reproducibility of results; reproducible research; reproducible science; software development.

PubMed Disclaimer

References

    1. Collange S., Defour D., Graillat S., Iakymchuk R. (2015). Numerical reproducibility for the parallel reduction on multi- and many-core architectures. Parallel Comput. 49, 83–97. 10.1016/j.parco.2015.09.001 - DOI
    1. Collberg C., Proebsting T. A. (2016). Repeatability in computer systems research. Commun. ACM 59, 62–69. 10.1145/2812803 - DOI
    1. Courtès L., Wurmus R. (2015). Reproducible and user-controlled software environments in HPC with Guix, in 2nd International Workshop on Reproducibility in Parallel Computing (RepPar) (Vienne: ).
    1. Crook S. M., Davison A. P., Plesser H. E. (2013). Learning from the past: approaches for reproducibility in computational neuroscience, in 20 Years of Computational Neuroscience, ed Bower J. M. (New York, NY: Springer; ), 73–102.
    1. Diethelm K. (2012). The limits of reproducibility in numerical simulation. Comput. Sci. Eng. 14, 64–72. 10.1109/mcse.2011.21 - DOI