Learning interpretable network dynamics via universal neural symbolic regression

Abstract

Discovering governing equations of complex network dynamics is a fundamental challenge in contemporary science with rich data, which can uncover the hidden patterns and mechanisms of the formation and evolution of complex phenomena in various fields and assist in decision-making. In this work, we develop a universal computational tool that can automatically, efficiently, and accurately learn the symbolic patterns of changes in complex system states by combining the excellent fitting capability of deep learning with the equation inference ability of pre-trained symbolic regression. We perform extensive and intensive experimental verifications on more than ten representative scenarios from fields such as physics, biochemistry, ecology, and epidemiology. The results demonstrate the remarkable effectiveness and efficiency of our tool compared to state-of-the-art symbolic regression techniques for network dynamics. The application to real-world systems including global epidemic transmission and pedestrian movements has verified its practical applicability. We believe that our tool can serve as a universal solution to dispel the fog of hidden mechanisms of changes in complex phenomena, advance toward interpretability, and inspire further scientific discoveries.

Fig. 1. The overall process of the LLC (Learning Law of Changes).

Observed data can be acquired from the initial experiments on the new scenario, including system states over time and topology, i.e. O. An interval selection strategy is to choose valid interval data and then we can get differentials, i.e. ${\stackrel{°}{X}}_i\left(t\right)$, through finite difference on X_i(t). By incorporating physical priors, the neural networks, i.e., ${\widehat{\mathit{Q}}}_{{\mathit{\theta }}_1}^{\left(self\right)}$ and ${\widehat{\mathit{Q}}}_{{\mathit{\theta }}_2}^{\left(inter\right)}$, are used to decouple the network dynamics signals and achieve variable reduction, continuing until the fitting requirements are met. Otherwise, the training is repeated. After obtaining well-fitted neural networks, we use symbolic regression techniques to efficiently parse their approximate white-box equations. Of course, if additional observed data are needed to support the discovery, the experimental design can be revisited until the satisfactory governing equations of network dynamics are obtained, breaking the loop.

Fig. 2. Results of inferring one-dimensional homogeneous network dynamics.

a Comparison of the accuracy on predictions (adjusted R² score) and discovered equations (Recall) for reconstructing dynamics from six scenarios, including Biochemical (Bio), Gene regulatory (Gene), Mutualistic Interaction (MI), Lotka-Volterra (LV), Neural (Neur), and Epidemic (Epi) dynamics. TPSINDy’s results are highly dominated by its choice of function terms while our LLC significantly outperforms the comparative methods across all network dynamics scenarios. b Comparison of the average execution time across all dynamics for various methods. Note that the TPSINDy requires strong priors, i.e., decomposability of self and interaction dynamics, as well as pre-defined orthogonal elementary function terms. In contrast, others are based on the same level of assumption that only requires decomposability. By combining with transformer-based pre-trained symbolic regression in our pipeline, we achieve a good balance between efficiency and accuracy. c The NED (Normalized Estimation Error) between the predictive results produced by the discovered governing equations and ground truth in the LV scenario. d Comparison of the fitting coefficients in governing equations discovered by various methods. e Comparison of state prediction curves for an individual node.

Fig. 3. Results of inferring the FitzHugh–Nagumo (FHN) dynamics.

a The fitting results of the first dimension (${\stackrel{°}{X}}_{i,1}\left(t\right)$) for a node by neural networks. b The decoupling results of the self dynamics for the first dimension for a node (${\widehat{\mathit{Q}}}_{{\mathit{\theta }}_1}^{\left(self\right)}$). c The decoupling results of the interaction dynamics for the first dimension for a node (${\widehat{\mathit{Q}}}_{{\mathit{\theta }}_2}^{\left(inter\right)}$). d The fitting results of the first dimension (${\stackrel{°}{X}}_{i,1}\left(t\right)$) for a node by neural networks. e Comparison of governing equations inferred by various methods. f Comparison of the normalized Euclidean distance (NED) between two trajectories, one generated from the inferred equations and the other from the true equations, with the horizontal axis representing the node index. g Comparison of the trajectories generated by the inferred and true equations on a Barabási–Albert network.

Fig. 4. Results of inferring the predator–prey (PP) system.

a Comparison of governing equations inferred by various methods. b The ground truth positions of a predator (square) and prey swarm (dots) over time. c The predictive positions generated by the governing equation inferred by the TPSINDy. d The predictive positions generated by the governing equation inferred by the LCC.

Fig. 5. Results of inferring the dynamics of chaotic systems.

a Comparison of governing equations inferred by our LCC under different initial conditions on the coupled Lorenz system. b Comparison of predictive states of an attractor with the same initial values, produced by equations inferred by our LLC under different initial conditions. c Coefficient errors between the equations inferred by each method on the Rössler system and the true equation. d Comparison of states of the same attractor, generated by the governing equations inferred by the TPSINDy, LCC, and LCC+TPSINDy on the Rössler system. e Bifurcation diagram of the Rössler system via the Poincaré section method, with the horizontal axis representing the parameter c (ranging from 1 to 6) and the vertical axis representing the states on the second dimension (X_i,2) of an attractor. The discovered equation exhibits the same period-doubling and chaotic phenomenon as the true equation. f Comparison of limit cycle at period-1, i.e., c = 2.5. g Comparison of chaos at c = 5.7.

Fig. 6. Results of inferring the dynamics of empirical systems, including global COVID-19 transmission and pedestrian dynamics.

a–d Comparison of the number of cases over time in various countries or regions generated by TPSINDy, LLC_each+TPSINDy, LLC_each, and LLC_total. e Comparison of governing equations inferred by various methods for four representative countries or regions. Note that the template in the LLC_each+TPSINDy is the induction of equations generated by our LLC for each node. f The real pedestrian movement trajectories over time, where the data before T is used for learning. g Inferred results on the pedestrian dynamics using a mainstream social force model (SFM). h Inferred results on the pedestrian dynamics using our LLC.