An adaptive weight ensemble approach to forecast influenza activity in an irregular seasonality context

Abstract

Forecasting influenza activity in tropical and subtropical regions, such as Hong Kong, is challenging due to irregular seasonality and high variability. We develop a diverse set of statistical, machine learning, and deep learning approaches to forecast influenza activity in Hong Kong 0 to 8 weeks ahead, leveraging a unique multi-year surveillance record spanning 32 epidemics from 1998 to 2019. We consider a simple average ensemble (SAE) of the top two individual models, and develop an adaptive weight blending ensemble (AWBE) that dynamically updates model contribution. All models outperform the baseline constant incidence model, reducing the root mean square error (RMSE) by 23%-29% and weighted interval score (WIS) by 25%-31% for 8-week ahead forecasts. The SAE model performed similarly to individual models, while the AWBE model reduces RMSE by 52% and WIS by 53%, outperforming individual models for forecasts in different epidemic trends (growth, plateau, decline) and during both winter and summer seasons. Using the post-COVID data (2023-2024) as another test period, the AWBE model still reduces RMSE by 39% and WIS by 45%. Our framework contributes to comparing and benchmarking models in ensemble forecasts, enhancing evidence for synthesizing multiple models in disease forecasting for geographies with irregular influenza seasonality.

Fig. 1. Influenza activity in Hong Kong.

A Influenza activity (ILI + ) in Hong Kong from 1998 to 2019, encompassing the influenza season, and the training and testing periods. Blue dotted vertical lines indicate the start of epidemics. B Influenza trends in a year Hong Kong. Epidemic week is defined as November of the preceding year to the October in the current year.

Fig. 2. Performance comparison of individual and ensemble models over the testing period by prediction horizon.

A–E referred to RMSE, MAE, WIS, SMAPE, and MAPE. Respectively. F showed the numerical value of the relative performance of metrics compared to the Baseline. Models: ARIMA Autoregressive Integrated Moving Average Model, GARCH Generalized AutoRegressive Conditional Heteroskedasticity Model, RF Random Forest, XGB Extreme Gradient Boosting, InTimePlus InceptionTime Plus Model, LSTM Long Short-Term Memory Network, GRU Gated Recurrent Neural Network, TSTPlus Transformer-based Framework for Multivariate Time Series Representation Learning Model, SAE Sample Average Ensemble model, NBE Normal Blending Ensemble model, AWAE Adaptive Weighted Average Ensemble model, AWBE Adaptive Weighted Blending Ensemble model.

Fig. 3. Trajectory of the forecasts of ensemble models.

Red, yellow and blue indicate the point forecasts of 0-week, 4-week and 8-week ahead, accompanying the 90% prediction interval by shaded area of corresponding colors.

Fig. 4. Model performance by epidemic phases and seasons.

All metrics are relative to the Baseline model and include RMSE, SMAPE, MAE, WIS, and MAPE. A Model performance for distinct epidemic trends, with black dashed lines representing the best individual models. B Different stages of Hong Kong flu data distinguished by color, with the gray dashed line signifying outbreak threshold. C Model performance by winter and summer seasons.

Fig. 5. Feature importance in nowcast, 4-week, and 8-week ahead forecasts.

Importance is measured by average regression coefficients in ARIMA and GARCH models, average feature importance in RF and XGB models, average saliency maps for LSTM and GRU models, and average permutation importance for TSTPlus and InTimePlus models. It should be noted that the numerical results of different comparison methods may not be directly comparable.

Fig. 6. The trajectory of forecast of adaptive ensemble models during the post-COVID period.

Red, yellow, blue and purple indicate the point forecasts of 0-week, 2-week, 4-week and 8-week ahead, accompanying the 90% prediction interval by shaded area of corresponding colors. A, B show the results of Adaptive Weighted Average Ensemble model (AWAE) and Adaptive Weighted Blending Ensemble (AWBE) respectively.