The utility of LASSO-based models for real time forecasts of endemic infectious diseases: A cross country comparison

Abstract

Introduction: Accurate and timely prediction for endemic infectious diseases is vital for public health agencies to plan and carry out any control methods at an early stage of disease outbreaks. Climatic variables has been identified as important predictors in models for infectious disease forecasts. Various approaches have been proposed in the literature to produce accurate and timely predictions and potentially improve public health response.

Methods: We assessed how the machine learning LASSO method may be useful in providing useful forecasts for different pathogens in countries with different climates. Separate LASSO models were constructed for different disease/country/forecast window with different model complexity by including different sets of predictors to assess the importance of different predictors under various conditions.

Results: There was a more apparent cyclicity for both climatic variables and incidence in regions further away from the equator. For most diseases, predictions made beyond 4 weeks ahead were increasingly discrepant from the actual scenario. Prediction models were more accurate in capturing the outbreak but less sensitive to predict the outbreak size. In different situations, climatic variables have different levels of importance in prediction accuracy.

Conclusions: For LASSO models used for prediction, including different sets of predictors has varying effect in different situations. Short term predictions generally perform better than longer term predictions, suggesting public health agencies may need the capacity to respond at short-notice to early warnings.

Keywords: Endemic infectious disease; LASSO; Real time forecast.

Graphical abstract

Fig. 1

Incidence and climatic (temperature and humidity) patterns of four representative diseases. (a) Chickenpox in Japan; (b) HFMD in Japan; (c) chickenpox in Thailand; (d) HFMD in Singapore. Year 01 corresponds to the year 2001 of the common era, et cetera.

Fig. 2

Wavelet power spectrum for (a) chickenpox in Japan; (b) HFMD in Japan; (c) chickenpox in Thailand; (d) HFMD in Singapore. For each panel, the power spectrum values are categorised by decile prior to plotting.

Fig. 3

Actual incidence (orange line) and forecasts (blue dots, dark red 95% projection interval, pink 70% projection interval) at four time points for each of the representative diseases. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Predicted cases against actual incidence at 1 week, 2 weeks, 4 weeks (1 month), and 8 weeks (2 months) for all prediction period for all diseases.

Fig. 5

Cumulative density functions (CDF) for observed time series for all diseases and the CDF for predicted time series at various forecast windows.

Fig. 6

Relative prediction error for models of different complexity as compared to the simplest incidence only model and absolute prediction error measured by MAPE for all models.