Credibility, Replicability, and Reproducibility in Simulation for Biomedicine and Clinical Applications in Neuroscience

Abstract

Modeling and simulation in computational neuroscience is currently a research enterprise to better understand neural systems. It is not yet directly applicable to the problems of patients with brain disease. To be used for clinical applications, there must not only be considerable progress in the field but also a concerted effort to use best practices in order to demonstrate model credibility to regulatory bodies, to clinics and hospitals, to doctors, and to patients. In doing this for neuroscience, we can learn lessons from long-standing practices in other areas of simulation (aircraft, computer chips), from software engineering, and from other biomedical disciplines. In this manuscript, we introduce some basic concepts that will be important in the development of credible clinical neuroscience models: reproducibility and replicability; verification and validation; model configuration; and procedures and processes for credible mechanistic multiscale modeling. We also discuss how garnering strong community involvement can promote model credibility. Finally, in addition to direct usage with patients, we note the potential for simulation usage in the area of Simulation-Based Medical Education, an area which to date has been primarily reliant on physical models (mannequins) and scenario-based simulations rather than on numerical simulations.

Keywords: Simulation-Based Medical Education; computational neuroscience; mechanistic modeling; model sharing; modeling and simulations; multiscale modeling; personalized and precision medicine; verification and validation.

FIGURE 1

Validation domain of a computational model for a range of input parameters and outputs (system responses). Red diamonds represent the validation set points, where comparisons between the computational model and applicable referents were conducted by discrete simulations performed in each referent sub-domain. The range of parameters evaluated establishes the Validation Domain (green ellipse), which will define the extent of the Application Domain (large blue ellipse) where model performance has established credibility. Applications of the model outside of the Application Domain lack validation and have lesser credibility.

FIGURE 2

Spectra to characterize biological aspects of interest. Classical engineering problems lie at the right side of each range, working with precise measures, strong expectations, detailed information and low uncertainty. Unfortunately, most neuroscience problems lie far to the left with weak measures, unclear phenomenology, sparse information and high uncertainty.

FIGURE 3

Mechanistic multiscale modeling process. (a) Causal explanation to be discovered. (b) Observables of phenomena to be explained. (c) Process (workflow) to identify and organize related information into descriptions; this process will involve simulation. (d) Meta-modeling workflows required for defining the extent of the final model. (e) Credible simulation which matches target phenomenon within some tolerance.