Bridging the gap between mechanistic biological models and machine learning surrogates

Abstract

Mechanistic models have been used for centuries to describe complex interconnected processes, including biological ones. As the scope of these models has widened, so have their computational demands. This complexity can limit their suitability when running many simulations or when real-time results are required. Surrogate machine learning (ML) models can be used to approximate the behaviour of complex mechanistic models, and once built, their computational demands are several orders of magnitude lower. This paper provides an overview of the relevant literature, both from an applicability and a theoretical perspective. For the latter, the paper focuses on the design and training of the underlying ML models. Application-wise, we show how ML surrogates have been used to approximate different mechanistic models. We present a perspective on how these approaches can be applied to models representing biological processes with potential industrial applications (e.g., metabolism and whole-cell modelling) and show why surrogate ML models may hold the key to making the simulation of complex biological systems possible using a typical desktop computer.

Fig 1. Schematic representation for training and using an ML-based surrogate model.

The mechanistic model is simulated (the top process connected by red arrows) to obtain the input-output pairs that are used to train the ML surrogate. This training stage (the middle process connected by orange arrows) is an average process in terms of speed. Its complexity will depend on the ML algorithm used, the complexity of the data preprocessing steps, and the quantity of training iterations needed to obtain a satisfactory accuracy. Once this is achieved, the ML model can be used for all future predictions, effectively approximating the mechanistic model while running several orders of magnitude faster. The green arrows at the bottom of the figure represent this final (fast) process.

Fig 2. Schematic representation of how active learning and ML surrogates can work together.

Initially, an ML model is trained on a set of data generated by some initial simulations of a mechanistic model (X_init, y_init), which are equivalent to (X, y) for this initial step. The ML model is used to make predictions (y_pred). The estimated error between the prediction of the mechanistic model (y) and that of the ML model (y_pred) is used to select a subset from all the possible input data that has not been used to make predictions using the mechanistic model in the past (X’). The mechanistic model is run using X’ as input to obtain a new set of input-output pairs (X, y), equivalent to the newly generated (X′, y′), that when included in the ML pipeline are expected to reduce the estimated error (y−y_pred).

Fig 3. An example of the DBTL pipeline where the metabolic or whole-cell models can be replaced by surrogate models.