Multi-Swarm Algorithm for Extreme Learning Machine Optimization

Abstract

There are many machine learning approaches available and commonly used today, however, the extreme learning machine is appraised as one of the fastest and, additionally, relatively efficient models. Its main benefit is that it is very fast, which makes it suitable for integration within products that require models taking rapid decisions. Nevertheless, despite their large potential, they have not yet been exploited enough, according to the recent literature. Extreme learning machines still face several challenges that need to be addressed. The most significant downside is that the performance of the model heavily depends on the allocated weights and biases within the hidden layer. Finding its appropriate values for practical tasks represents an NP-hard continuous optimization challenge. Research proposed in this study focuses on determining optimal or near optimal weights and biases in the hidden layer for specific tasks. To address this task, a multi-swarm hybrid optimization approach has been proposed, based on three swarm intelligence meta-heuristics, namely the artificial bee colony, the firefly algorithm and the sine-cosine algorithm. The proposed method has been thoroughly validated on seven well-known classification benchmark datasets, and obtained results are compared to other already existing similar cutting-edge approaches from the recent literature. The simulation results point out that the suggested multi-swarm technique is capable to obtain better generalization performance than the rest of the approaches included in the comparative analysis in terms of accuracy, precision, recall, and f1-score indicators. Moreover, to prove that combining two algorithms is not as effective as joining three approaches, additional hybrids generated by pairing, each, two methods employed in the proposed multi-swarm approach, were also implemented and validated against four challenging datasets. The findings from these experiments also prove superior performance of the proposed multi-swarm algorithm. Sample code from devised ELM tuning framework is available on the GitHub.

Keywords: extreme learning machine; hybridization; machine learning; meta-heuristic algorithms; multi-swarm algorithm; swarm intelligence.

Figure 1

Overview for the proposed ELM-MS-AFS approach.

Figure 2

Distribution of classes in Diabetes, Disease, Iris, Wine, and Wine Quality datasets before split.

Figure 3

Distribution of classes in Satellite and Shuttle datasets with predetermined training and testing subsets.

Figure 4

Graphs for convergence speed evaluation on seven observed datasets for 30, 60, and 90 neurons, for the proposed method vs. other approaches.

Figure 4

Graphs for convergence speed evaluation on seven observed datasets for 30, 60, and 90 neurons, for the proposed method vs. other approaches.

Figure 5

Generated confusion matrices and PR curves for some datasets by ELM-MS-AFS.

Figure 6

Distribution of classes in NSL-KDD dataset with predetermined training and testing subsets.

Figure 7

Graphs for convergence speed evaluation on four observed datasets for 30, 60, and 90 neurons, for the proposed ELM-MS-AFS vs. other hybrid approaches.

Figure 8

Generated PR curves for NSL-KDD dataset by hybrid methods.