Machine Learning of Time Series Using Time-Delay Embedding and Precision Annealing

Abstract

Tasking machine learning to predict segments of a time series requires estimating the parameters of a ML model with input/output pairs from the time series. We borrow two techniques used in statistical data assimilation in order to accomplish this task: time-delay embedding to prepare our input data and precision annealing as a training method. The precision annealing approach identifies the global minimum of the action ($-log\left[P\right]$). In this way, we are able to identify the number of training pairs required to produce good generalizations (predictions) for the time series. We proceed from a scalar time series $s\left(t_n\right);t_n=t_0+n\Delta t$ and, using methods of nonlinear time series analysis, show how to produce a $D_E>1$-dimensional time-delay embedding space in which the time series has no false neighbors as does the observed $s\left(t_n\right)$ time series. In that $D_E$-dimensional space, we explore the use of feedforward multilayer perceptrons as network models operating on $D_E$-dimensional input and producing $D_E$-dimensional outputs.