Comparing models for early warning systems of neglected tropical diseases

Abstract

Background: Early warning systems (EWS) are management tools to predict the occurrence of epidemics of infectious diseases. While climate-based EWS have been developed for malaria, no standard protocol to evaluate and compare EWS has been proposed. Additionally, there are several neglected tropical diseases whose transmission is sensitive to environmental conditions, for which no EWS have been proposed, though they represent a large burden for the affected populations.

Methodology/principal findings: In the present paper, an overview of the available linear and non-linear tools to predict seasonal time series of diseases is presented. Also, a general methodology to compare and evaluate models for prediction is presented and illustrated using American cutaneous leishmaniasis, a neglected tropical disease, as an example. The comparison of the different models using the predictive R(2) for forecasts of "out-of-fit" data (data that has not been used to fit the models) shows that for the several linear and non-linear models tested, the best results were obtained for seasonal autoregressive (SAR) models that incorporate climatic covariates. An additional bootstrapping experiment shows that the relationship of the disease time series with the climatic covariates is strong and consistent for the SAR modeling approach. While the autoregressive part of the model is not significant, the exogenous forcing due to climate is always statistically significant. Prediction accuracy can vary from 50% to over 80% for disease burden at time scales of one year or shorter.

Conclusions/significance: this study illustrates a protocol for the development of EWS that includes three main steps: (i) the fitting of different models using several methodologies, (ii) the comparison of models based on the predictability of "out-of-fit" data, and (iii) the assessment of the robustness of the relationship between the disease and the variables in the model selected as best with an objective criterion.

Figure 1. Time Series.

(A) Square root Transformed ACL Cases in Costa Rica. (B) Mean Temperature in Costa Rica. (C) MEI.

Figure 2. Multidimensional plots for the square root transformed ACL cases (y_t) as function of lagged components and climatic covariates.

(A) Autoregressive (y_t−1) and Seasonal (y_t−12) components. (B) Seasonal (y_t−12) and Autoregressive Seasonal (y_t−13) components. (C) Autoregressive component (y_t−1) and Temperature (lag 4, T_t−4). (D) Autoregressive component (y_t−1) and MEI (lag 13, MEI_t−13).

Figure 3. Bootstrap Experiment.

(A) 95% Confidence intervals for the parameters of the best model. AR stands for the autoregressive component of the model (φ₁); ARseas for the seasonal autoregressive component of the model (φ₁₂); VAR for the variance of the residuals (σ_ε²); MEI and T4 for the parameter for MEI at lag 13 (α) and Temperature at lag 4 (γ), respectively. Black signs are 95% confidence intervals using values from the sub-sample when the model is selected as best, and blue including all the bootstrap samples. The structure of the best model can be seen in Protocol S1. (B) Predictive R ² and the 95% confidence intervals, indicated by stars, for the bootstrapped best model and prediction interval.