Multi-Perspective Anomaly Detection

Abstract

Anomaly detection is a critical problem in the manufacturing industry. In many applications, images of objects to be analyzed are captured from multiple perspectives which can be exploited to improve the robustness of anomaly detection. In this work, we build upon the deep support vector data description algorithm and address multi-perspective anomaly detection using three different fusion techniques, i.e., early fusion, late fusion, and late fusion with multiple decoders. We employ different augmentation techniques with a denoising process to deal with scarce one-class data, which further improves the performance (ROC AUC =80%). Furthermore, we introduce the dices dataset, which consists of over 2000 grayscale images of falling dices from multiple perspectives, with 5% of the images containing rare anomalies (e.g., drill holes, sawing, or scratches). We evaluate our approach on the new dices dataset using images from two different perspectives and also benchmark on the standard MNIST dataset. Extensive experiments demonstrate that our proposed multi-perspective approach exceeds the state-of-the-art single-perspective anomaly detection on both the MNIST and dices datasets. To the best of our knowledge, this is the first work that focuses on addressing multi-perspective anomaly detection in images by jointly using different perspectives together with one single objective function for anomaly detection.

Keywords: anomaly detection; data fusion; deep learning; multi-perspective; novelty detection; one-class learning.

Figure 1

Illustration of the convolutional autoencoder used for multi-perspective anomaly detection. There are five layers each in the encoder and decoder block. Each layer can be identified with a particular name (“conv” for convolution, “tconv” for transpose convolution, and “fc” for fully connected) and the size that it outputs. The arrow towards Deep SVDD portrays that this is used as the feature space where the hypersphere is found. Figure partly generated with LeNail’s schematics [32].

Figure 2

Given two perspectives of the same dice, this figure illustrates the training and fusion techniques that we use in this work. Left column: The first stage comprises of the auto encoder training. Right column: The weights of the encoder are then further trained alone according to Equation (1). (a) With an early fusion the two perspectives were stacked channel-wise. Hence the output of the encoder comprises information of both perspectives inherently. (b,c) In contrast to that, two outputs were averaged in a late fusion. (c) The last row shows the utilization of two decoders.

Figure 3

(a) Four different types of anomalies were generated for dices: Drilling (17% occurances in test dataset), missing dots (33%), sawing (17%) and scratching (33%). (b) The measurement setup free-fall [34] where all 27 cameras are directed to the center of the globe.

Figure 4

Four additional training sets are created that extend the original data with augmentation. Each set consists of 4000 channel wise stacked images (400,400,2). (a) The first set contains all augmentation strategies, i.e., erasing patches, changing image constituents like brightness and altering geometry like flipping. The other training sets exclude particular strategies: (b) w/o erased patching, (c) w/o change of image constitution and (d) w/o geometrical rotations and flipping.

Figure 5

Most normal (a) and most anomalous (b) samples determined by multi-perspective denoising fusion technique for multi-perspective MNIST [9] dataset. In this dataset, samples representing zero are considered to be from non-anomalous class and other digits are considered to be from the anomalous class. (c) illustrates the reconstructed output for examples from the test set.