Novel hybrid data-driven modeling based on feature space reconstruction and multihead self-attention gated recurrent unit: applied to PM2.5 concentrations prediction

Abstract

In response to the problem of neglecting the periodic and global characteristics of sequence data when predicting PM2.5 concentrations via machine learning models, a PM2.5 concentrations prediction model based on feature space reconstruction and multihead self-attention gated recurrent unit (FSR-MSAGRU) is proposed in this study. First, the raw sequence data are subjected to frequency spectrum analysis to determine the period value of the PM2.5 sequence data. Subsequently, the seasonal trend decomposition procedure based on loess (STL) is employed to capture the periodicity and trend information in the PM2.5 sequence data. Then, the feature space of the PM2.5 sequence data is reconstructed using the raw PM2.5 sequence data, decomposed seasonal components, trend components, and residual components. Finally, the reconstructed feature data are input into multihead self-attention gated recurrent unit (MSAGRU) with the ability to capture global feature information to predict PM2.5 concentrations. Favorable prediction results were attained by the proposed FSR-MSAGRU model across 6 distinct experimental datasets, with a PCC exceeding 0.98 and a decrease in the prediction accuracy metric SMAPE of at least 68% compared to that of the GRU model. Comparative experimental results with 13 reference models demonstrate that the proposed model exhibits better prediction performances and stronger generalization abilities.

Keywords: Feature space reconstruction; Gated recurrent unit; Machine learning; Multihead Self-attention; PM2.5 concentration prediction.

Fig. 1

Locations of cities for data collection.

Fig. 2

Distribution of the dataset samples.

Fig. 3

Flowchart of the data processing.

Fig. 4

Frequency spectrum of PM2.5 in different datasets.

Fig. 5

Reconstructed feature spaces in the different datasets.

Fig. 6

Framework of the FSR-MSAGRU model.

Fig. 7

Comparison curves of the ablation experiment prediction results.

Fig. 8

Comparison curves and scatter plots of the single models, hybrid models, and FSR-MSAGRU model prediction results (Lanzhou).

Fig. 9

Performance metrics of the different models on various datasets.

Fig. 10

Taylor diagram of the prediction results of the different models on the various datasets.