Data-driven discovery of coordinates and governing equations

Abstract

The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam's razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.

Keywords: deep learning; dynamical systems; machine learning; model discovery.

Fig. 1.

Schematic of the SINDy autoencoder method for simultaneous discovery of coordinates and parsimonious dynamics. (A) An autoencoder architecture is used to discover intrinsic coordinates $\mathbf{z}$ from high-dimensional input data $\mathbf{x}$. The network consists of 2 components: an encoder $\phi \left(\mathbf{x}\right)$, which maps the input data to the intrinsic coordinates $\mathbf{z}$, and a decoder $\psi \left(\mathbf{z}\right)$, which reconstructs $\mathbf{x}$ from the intrinsic coordinates. (B) A SINDy model captures the dynamics of the intrinsic coordinates. The active terms in the dynamics are identified by the nonzero elements in $\mathbf{\Xi }$, which are learned as part of the NN training. The time derivatives of $\mathbf{z}$ are calculated using the derivatives of $\mathbf{x}$ and the gradient of the encoder $\phi$. Inset shows the pointwise loss function used to train the network. The loss function encourages the network to minimize both the autoencoder reconstruction error and the SINDy loss in $\mathbf{z}$ and $\mathbf{x}$. $L_1$ regularization on $\mathbf{\Xi }$ is also included to encourage parsimonious dynamics.

Fig. 2.

Discovered models for examples. (A–C) Equations, SINDy coefficients $\mathbf{\Xi }$, and attractors for Lorenz (A), reaction–diffusion (B), and nonlinear pendulum (C) systems.

Fig. 3.

Model results on the high-dimensional Lorenz example. (A) Trajectories of the chaotic Lorenz system ($\mathbf{z}\left(t\right)\in {\mathbb{R}}^3$) are used to create a high-dimensional dataset ($\mathbf{x}\left(t\right)\in {\mathbb{R}}^{128}$). (B) The spatial modes are created from the first 6 Legendre polynomials and the temporal modes are the variables in the Lorenz system and their cubes. The spatial and temporal modes are combined to create the high-dimensional dataset via [6]. (C and D) The equations, SINDy coefficients $\mathbf{\Xi }$, and attractors for the original Lorenz system and a dynamical system discovered by the SINDy autoencoder. The attractors are constructed by simulating the dynamical system forward in time from a single initial condition. (E) Applying a suitable variable transformation to the system in D reveals a model with the same sparsity pattern as the original Lorenz system. The parameters are close in value to the original system, with the exception of an arbitrary scaling, and the attractor has a similar structure to the original system.