Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection

Abstract

We present an unsupervised method to detect anomalous time series among a collection of time series. To do so, we extend traditional Kernel Density Estimation for estimating probability distributions in Euclidean space to Hilbert spaces. The estimated probability densities we derive can be obtained formally through treating each series as a point in a Hilbert space, placing a kernel at those points, and summing the kernels (a "point approach"), or through using Kernel Density Estimation to approximate the distributions of Fourier mode coefficients to infer a probability density (a "Fourier approach"). We refer to these approaches as Functional Kernel Density Estimation for Anomaly Detection as they both yield functionals that can score a time series for how anomalous it is. Both methods naturally handle missing data and apply to a variety of settings, performing well when compared with an outlyingness score derived from a boxplot method for functional data, with a Principal Component Analysis approach for functional data, and with the Functional Isolation Forest method. We illustrate the use of the proposed methods with aviation safety report data from the International Air Transport Association (IATA).

Keywords: anomaly detection; kernel density estimation; missing data; time series; unsupervised learning.

Figure 1

A visual depiction of the Point method. The curves are time series in a Hilbert space $\mathcal{H}$ but after applying KDE, there is a score associated to each point in $\mathcal{H}$. In the cartoon, curves 1 and 2 are similar and curve 3 is anomalous. (Left): the time series. (Right): a representation of them with associated scores in the color scale. In reality, the space is infinite dimensional and this is only a conceptual illustration.

Figure 2

Plot of 63 normal curves and the 7 anomalous curves $C_i\left(t\right)$, $i=1,\dots ,7$. Left: un-normalized. Right: normalized.

Figure 3

Histogram of scores for Point and Fourier methods for Type A Point (top-left), Type A Fourier(top-right), Type B Point (bottom-left) and Type B Fourier (bottom-right). The dashed vertical line represents the division we chose between anomalous (left of line) and normal (right of line). The Sturges estimate was used to set bin widths [30].

Figure 4

Plots of the time series for Type A and Type B events. Anomalous are dotted curves with markers in the legend; normal curves are solid black curves.