Surrogate modeling and control of medical digital twins

Abstract

The vision of personalized medicine is to identify interventions that maintain or restore a person's health based on their individual biology. Medical digital twins, computational models that integrate a wide range of health-related data about a person and can be dynamically updated, are a key technology that can help guide medical decisions. Such medical digital twin models can be high-dimensional, multi-scale, and stochastic. To be practical for healthcare applications, they often need to be simplified into low-dimensional surrogate models that can be used for optimal design of interventions. This paper introduces surrogate modeling algorithms for the purpose of optimal control applications. As a use case, we focus on agent-based models (ABMs), a common model type in biomedicine for which there are no readily available optimal control algorithms. By deriving surrogate models that are based on systems of ordinary differential equations, we show how optimal control methods can be employed to compute effective interventions, which can then be lifted back to a given ABM. The relevance of the methods introduced here extends beyond medical digital twins to other complex dynamical systems.

Fig 1.. Summary of the key steps involved in using ODE surrogate models for control.

For an ABM associated with a control problem, we first create an ODE surrogate (Step 1). Next, we apply control techniques to the ODE system (Step 2) before lifting the control solution back to the original ABM (Step 3). The snapshots in the ABM panel depict simulations performed in NetLogo[34] of tumor[35], slime mold[36], and wolf sheep predation models[37].

Fig 2.. Correspondence between ABM and ODE model components.

The aggregation of agents by type or attributes characterizes the state variables of an ODE approximation. Similarly, all interactions, events, and rules in the ABM characterize the processes of an ODE model. Depending on the ABM’s structure, its environment can be transformed into state variables or contribute to the processes of an ODE approximation.

Fig 3.. Comparison of effectiveness of different ODE surrogate models for solving the sheep-wolves-grass ABM control problem.

The black cross marks the near-optimal solution (${\kappa }_2$ = 0.83% and ${\kappa }_3$ = 0.45% per time step) for the sheep-wolves-grass ABM control problem as determined by a grid search (with a step of 0.0001 in both dimensions). Orange dots indicate suboptimal control solutions within one standard deviation from the target (a steady state with 50% fewer wolves and 10% more sheep compared to the original steady state). Blue and red dots show the control parameter values associated with the ODE surrogate models that have been calibrated against datasets I and II and datasets I-V, respectively. The best solutions were obtained for surrogate models parameterized with datasets containing control information (III-V). However, all four of these surrogate models (red dots) identified control solutions equally distant from the optimal one.

Fig 4.. Comparison of effectiveness of different ODE surrogate models for solving the metabolic pathway ABM control problem.

The red square shows the optimal inflow point and the corresponding mean loss function value as determined for the ABM by a grid search between 0 and 1.0 with a step size of 0.1, where in each step 100 simulations runs were averaged. The red line highlights the mean of each of the 100 simulation runs of the ABM and the orange line the 75% confidence band. Circles denote the predicted optimal inflow and corresponding loss function value for each ODE surrogate. ODE models that did not exhibit a minimum within the 0 to 1.0 domain have their domain of integrability shown with a line. The line depicts the range of loss function values predicted by the approximation. The S-system I performed worst, as it could only be integrated between 0.8 and 1.0, and in that range predicted loss function values between 8 and 9. While S-system C, Quad I, and Linear I, all resulted in models with a larger domain over which they could be integrated, neither had a minimum within their respective domains. GMA I was the ODE surrogate that best predicted an optimal inflow of substrate closest to the ABM and Mech. I best predicted the loss function value of the ABM at the optimal inflow point.