A Hybrid Scattering Transform for Signals with Isolated Singularities

Abstract

The scattering transform is a wavelet-based model of Convolutional Neural Networks originally introduced by S. Mallat. Mallat's analysis shows that this network has desirable stability and invariance guarantees and therefore helps explain the observation that the filters learned by early layers of a Convolutional Neural Network typically resemble wavelets. Our aim is to understand what sort of filters should be used in the later layers of the network. Towards this end, we propose a two-layer hybrid scattering transform. In our first layer, we convolve the input signal with a wavelet filter transform to promote sparsity, and, in the second layer, we convolve with a Gabor filter to leverage the sparsity created by the first layer. We show that these measurements characterize information about signals with isolated singularities. We also show that the Gabor measurements used in the second layer can be used to synthesize sparse signals such as those produced by the first layer.

Keywords: deep learning; scattering transforms; sparsity; time-frequency analysis; wavelets.

Fig. 1:

Wavelet filter used in the first layer

Fig. 2:

Wavelets sparsify piecewise polynomials on the interval [0, 1024h].

Fig. 3:

Gabor filters used in the second layer (real parts)

Fig. 4:

Sparse signals reconstructed up to a global reflections, translations, and sign changes. Originals signals are on the left and reconstructed signals are on the right.