Kernel learning for robust dynamic mode decomposition: linear and nonlinear disambiguation optimization

Abstract

Research in modern data-driven dynamical systems is typically focused on the three key challenges of high dimensionality, unknown dynamics and nonlinearity. The dynamic mode decomposition (DMD) has emerged as a cornerstone for modelling high-dimensional systems from data. However, the quality of the linear DMD model is known to be fragile with respect to strong nonlinearity, which contaminates the model estimate. By contrast, sparse identification of nonlinear dynamics learns fully nonlinear models, disambiguating the linear and nonlinear effects, but is restricted to low-dimensional systems. In this work, we present a kernel method that learns interpretable data-driven models for high-dimensional, nonlinear systems. Our method performs kernel regression on a sparse dictionary of samples that appreciably contribute to the dynamics. We show that this kernel method efficiently handles high-dimensional data and is flexible enough to incorporate partial knowledge of system physics. It is possible to recover the linear model contribution with this approach, thus separating the effects of the implicitly defined nonlinear terms. We demonstrate our approach on data from a range of nonlinear ordinary and partial differential equations. This framework can be used for many practical engineering tasks such as model order reduction, diagnostics, prediction, control and discovery of governing laws.

Keywords: kernel methods; machine learning; modal decomposition; system identification.

Figure 1.

Learning regression models in linear (a) and nonlinear (b) feature spaces. Our approach disambiguates linear and nonlinear model contributions to accurately extract local linear models. (Online version in colour.)

Figure 2.

The linear and nonlinear disambiguation optimization (LANDO) framework. Training data in the form of snapshot pairs are collected from either simulation or experiment in (1). The data are organized into matrices in (2). In (3), an appropriate kernel is defined, which can be informed by expert knowledge of the underlying physics of the system or through cross-validation. In (4), a sparse dictionary of basis elements is constructed from the training samples, and in (5), the regression problem is solved. Finally, in (6), an interpretable model is extracted. (Online version in colour.)

Figure 3.

Schematic relationships between different models for $N\gg n,m\gg \tilde{m}$. An explicit model (e.g. SINDy) produces explicit weights that connect $N$ features to $n$ outputs. A kernel model uses fewer weights but the relationships between variables are stored implicitly. The dictionary-based kernel model selects the most active samples and therefore uses fewer weights still. (Online version in colour.)

Figure 4.

Comparing the ALD dictionaries computed by the original KRLS algorithm, the Cholesky updating variant and a batch offline algorithm when applied to a solution of the viscous Burgers’ equation. The kernel here is quadratic and the sparsity parameter is $\nu =0.1$. (a) The computed distance of each sample from the span of the current dictionary, which determines whether the current sample should be added to the dictionary. (b) Plots the growth of the dictionary as more samples are considered. The original KRLS algorithm misidentifies dictionary elements and the corresponding dictionary is larger than necessary. (Online version in colour.)

Figure 5.

A comparison of learned linear operators for dynamical systems and PDEs. In the linear operators, red represents positive quantities whereas blue represents negative quantities. The problem sizes are provided in §5. (Online version in colour.)

Figure 6.

Kernel learning of the Lorenz system. We compare the learned models and predicted trajectories for linear, quadratic and Gaussian kernels. The training data are discrete-time snapshots of the state ${[x\hspace{0.17em}y\hspace{0.17em}z]}^T$ and the corresponding velocity measurements. The top row shows the models’ reconstructions of the training data, the middle row shows the predicted trajectory from a different initial conditions, and the bottom row shows the learned linear model near the equilibrium point $\overline{\mathit{x}}={[-\sqrt{\beta (\gamma -1)}\hspace{0.17em}-\sqrt{\beta (\gamma -1)}\hspace{0.17em}\gamma -1]}^T$, which is indicated by square. The parameter values are $\varsigma =10$, $\gamma =28$ and $\beta =8/3$ and the initial conditions are represented by circle. (Online version in colour.)

Figure 7.

Learning the natural frequencies of coupled oscillators. The training data are generated from a forced Kuramoto model and are illustrated in (a). The LANDO framework extracts the natural frequencies of the model. These learned natural frequencies are compared with the true natural frequencies in (b) and the frequencies learned by a linear (DMD) model; because there are 2000 oscillators, only a handful of frequencies are plotted. (Online version in colour.)

Figure 8.

Learning the spectrum of the viscous Burgers’ equation. A typical simulation is illustrated in (a) with the initial condition highlighted in green. The algorithm is trained on discrete time snapshots ${\mathit{y}}_j={\mathit{x}}_{j+1}$; velocity measurements $\dot{\mathit{x}}$ are not used in the training set. The figures in (b) indicate that the algorithm accurately learns the eigenvalues, ${\lambda }_n=-\nu n^2{\pi }^2$, and eigenfunctions, $\sin ({\lambda }_{n}x)$ and $\cos ({\lambda }_{n}x)$, of the linearized operator at the state $u\equiv 0$. (Online version in colour.)

Figure 9.

Learning the spectrum of the Kuramoto–Sivashinsky equation with a domain size $L=14\pi$, for which the system exhibits chaotic dynamics. A typical simulation is illustrated in (a). The algorithm is trained on discrete time snapshots ${\mathit{y}}_j={\dot{\mathit{x}}}_j$. (b) The algorithm accurately learns the eigenvalues of the linearized operator at the state $u\equiv 0$. The size of the markers of the LANDO eigenvalues correspond to the average projection of the training data onto the associated eigenvectors. (Online version in colour.)