Bidimensionally partitioned online sequential broad learning system for large-scale data stream modeling

Abstract

Incremental broad learning system (IBLS) is an effective and efficient incremental learning method based on broad learning paradigm. Owing to its streamlined network architecture and flexible dynamic update scheme, IBLS can achieve rapid incremental reconstruction on the basis of the previous model without the entire retraining from scratch, which enables it adept at handling streaming data. However, two prominent deficiencies still persist in IBLS and constrain its further promotion in large-scale data stream scenarios. Firstly, IBLS needs to retain all historical data and perform associated calculations in the incremental learning process, which causes its computational overhead and storage burden to increase over time and as such puts the efficacy of the algorithm at risk for massive or unlimited data streams. Additionally, due to the random generation rule of hidden nodes, IBLS generally necessitates a large network size to guarantee approximation accuracy, and the resulting high-dimensional matrix calculation poses a greater challenge to the updating efficiency of the model. To address these issues, we propose a novel bidimensionally partitioned online sequential broad learning system (BPOSBLS) in this paper. The core idea of BPOSBLS is to partition the high-dimensional broad feature matrix bidimensionally from the aspects of instance dimension and feature dimension, and consequently decompose a large least squares problem into multiple smaller ones, which can then be solved individually. By doing so, the scale and computational complexity of the original high-order model are substantially diminished, thus significantly improving its learning efficiency and usability for large-scale complex learning tasks. Meanwhile, an ingenious recursive computation method called partitioned recursive least squares is devised to solve the BPOSBLS. This method exclusively utilizes the current online samples for iterative updating, while disregarding the previous historical samples, thereby rendering BPOSBLS a lightweight online sequential learning algorithm with consistently low computational costs and storage requirements. Theoretical analyses and simulation experiments demonstrate the effectiveness and superiority of the proposed algorithm.

Keywords: Big data modeling; Broad learning system; Matrix partitioning; Online sequential learning; Partitioned recursive least squares.

Fig. 1

The structure of BLS.

Algorithm 1

BPOSBLS.

Fig. 2

The structure of BPOSBLS.

Fig. 3

Comparison of the online prediction RMSE between IBLS and BPOSBLS on (a) Mackey-Glass, (b) Logistic, (c) Henon, (d) Santa Fe, (e) ESTSP.

Fig. 4

Comparison of the online learning times between IBLS and BPOSBLS on (a) Mackey-Glass, (b) Logistic, (c) Henon, (d) Santa Fe, (e) ESTSP.

Fig. 5

Prediction error of BPOSBLS under different values of (, s). (a) Prediction RMSE in terms of . (b) Prediction RMSE in terms of s.

Fig. 6

Learning time of BPOSBLS under different values of (, s). (a) Learning time in terms of . (b) Learning time in terms of s.