Best Practices for Making Reproducible Biochemical Models

Abstract

Like many scientific disciplines, dynamical biochemical modeling is hindered by irreproducible results. This limits the utility of biochemical models by making them difficult to understand, trust, or reuse. We comprehensively list the best practices that biochemical modelers should follow to build reproducible biochemical model artifacts-all data, model descriptions, and custom software used by the model-that can be understood and reused. The best practices provide advice for all steps of a typical biochemical modeling workflow in which a modeler collects data; constructs, trains, simulates, and validates the model; uses the predictions of a model to advance knowledge; and publicly shares the model artifacts. The best practices emphasize the benefits obtained by using standard tools and formats and provides guidance to modelers who do not or cannot use standards in some stages of their modeling workflow. Adoption of these best practices will enhance the ability of researchers to reproduce, understand, and reuse biochemical models.

Keywords: COmputational Modeling in BIology NEtwork; FAIR principles; biochemical models; modeling; reproducibility; standards; systems biology.

Figure 1.. Practical Recommendations of Tools for Reproducible Modeling across All Stages of the Typical Biochemical Modeling Workflow

(A) A typical workflow that creates and uses a dynamical model: in “aggregate data,” a modeler collects data from papers, public data sources and/or private experiments; in “construct model,” they use the data, their biological knowledge, assumptions, and modeling methods to create a model; in “estimate parameters,” the modeler produces a complete and self-consistent set of input parameters from the data; in “simulate model,” the modeler integrates the model over time; in “store and analyze results,” they store simulation results and analyze them; in “verify & validate model,” the modeler ensures that the model and its predictions are consistent with experimental data; in “document artifacts,” the modeler annotates and provides human-readable descriptions (tan rectangles) for all model artifacts from each stage; in “package artifacts and documentation,” they combine all model artifacts and documentation into archive(s) to be shared publicly, and in “publish and disseminate,” the modeler publishes their novel scientific findings and shares the archive(s) by depositing them in open-source repositories that independent researchers can access to reproduce, understand, and reuse the model. Black arrows indicate the transitions between workflow stages. (B) Software tools and data formats for reproducible modeling: Tools and data formats that enhance reproducibility are listed in a diagram that parallels the workflow in (A). These tools and data formats are split into recommendations for standards-based and general-purpose approaches to modeling, as presented in the text. Tools that are useful in multiple modeling stages are listed in those stages. A table with links to the tools shown in Figure 1 is included in the Supplemental Information (see Table S1).”