Noniterative Sparse LS-SVM Based on Globally Representative Point Selection

Abstract

A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.