Exploring feature sparsity for out-of-distribution detection

Abstract

Out-of-distribution (OOD) detection is a crucial problem in practice, especially, for the safe deployment of machine learning models in industrial settings. Previous work has used free energy as a score function and proposed a fine-tuning method that utilized OOD data in the training phase of the classification model, which achieves a higher performance on the OOD detection task compared with traditional methods. One key drawback, however, is that the loss function parameters are highly dependent on involved datasets, which means it cannot be dynamically adapted and implemented in others settings; in other words, the general ability of the energy score is considerably limited. In this work, our point of departure is to enlarge distinguishability between in-distribution features and OOD data. Consequently, we present a simple yet effective sparsity-regularized (SR) tuning framework for this purpose. Our framework has two types of workflows depending on if external OOD data is available, the complexity of the original training loss is sharply reduced by adopting this modification, meanwhile, the adapted ability and detection performance are enhanced. Also, we contribute a mini dataset as a light and efficient alternative of the previous large-scale one. In the experiments, we verify the effectiveness of our framework in a wide range of typical datasets along with common network architectures.

Fig. 1

The optimization process of our Sparsity-Regularized framework. We exert a sparsification for the ID feature when fine-tine the model, and the solid paths indicate the flow of , and the dashed paths indicate the flow of . Note that the dashed part is optional, and the total tuning loss is when there is no available; otherwise, .

Algorithm 1

Sparsity-regularized framework.

Fig. 2

The visualization of detection. The larger the intersection area is, the worse the OOD detection performance is in the above six graphs.

Fig. 3

The original data pool of “tiny”. Samples labelled with “in” are too similar to ID data and must be removed, while samples labelled with “out” can be retained as elements of “tiny”.

Fig. 4

ROC curves for OOD detection and classifier accuracy among three models. We can observe that the sparsity-regularized method enhances detection performance without any degradation of classification accuracy.