Next generation reservoir computing

Abstract

Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metaparameters, and provides interpretable results. Here, we demonstrate that nonlinear vector autoregression excels at reservoir computing benchmark tasks and requires even shorter training data sets and training time, heralding the next generation of reservoir computing.

Fig. 1. A traditional RC is implicit in an NG-RC.

(top) A traditional RC processes time-series data associated with a strange attractor (blue, middle left) using an artificial recurrent neural network. The forecasted strange attractor (red, middle right) is a linear weight of the reservoir states. (bottom) The NG-RC performs a forecast using a linear weight of time-delay states (two times shown here) of the time series data and nonlinear functionals of this data (quadratic functional shown here).

Fig. 2. Forecasting a dynamical system using the NG-RC.

True (a) and predicted (e) Lorenz63 strange attractors. b–d Training data set with overlayed predicted behavior with α = 2.5 × 10⁻⁶. The normalized root-mean-square error (NRMSE) over one Lyapunov time during the training phase is 1.06 ± 0.01 × 10⁻⁴, where the uncertainty is the standard error of the mean. f–h True (blue) and predicted datasets during the forecasting phase (NRMSE = 2.40 ± 0.53 × 10⁻³).

Fig. 3. Forecasting the double-scroll system using the NG-RC.

True (a) and predicted (e) double-scroll strange attractors. b–d Training data set with overlayed predicted behavior. f–h True (blue) and predicted datasets during the forecasting phase (NRMSE = 4.5 ± 1.0 × 10⁻³).

Fig. 4. Inference using an NG-RC.

a–c Lorenz63 variables during the training phase (blue) and prediction (c, red). The predictions overlay the training data in (c), resulting in a purple trace (NRMSE = 9.5 ± 0.1 × 10⁻³ using α = 0.05). d–f Lorenz63 variables during the testing phase, where the predictions overlay the training data in (f), resulting in a purple trace (NRMSE = 1.75 ± 0.3 × 10⁻²).