Early detection of West Nile virus in France: quantitative assessment of syndromic surveillance system using nervous signs in horses

Abstract

West Nile virus (WNV) is a growing public health concern in Europe and there is a need to develop more efficient early detection systems. Nervous signs in horses are considered to be an early indicator of WNV and, using them in a syndromic surveillance system, might be relevant. In our study, we assessed whether or not data collected by the passive French surveillance system for the surveillance of equine diseases can be used routinely for the detection of WNV. We tested several pre-processing methods and detection algorithms based on regression. We evaluated system performances using simulated and authentic data and compared them to those of the surveillance system currently in place. Our results show that the current detection algorithm provided similar performances to those tested using simulated and real data. However, regression models can be easily and better adapted to surveillance objectives. The detection performances obtained were compatible with the early detection of WNV outbreaks in France (i.e. sensitivity 98%, specificity >94%, timeliness 2·5 weeks and around four false alarms per year) but further work is needed to determine the most suitable alarm threshold for WNV surveillance in France using cost-efficiency analysis.

Keywords: Early detection; West Nile virus; horses; nervous signs; syndromic surveillance.

Fig. 1.

Four time series used. TS0, raw data; TS1, only the cases with no positive laboratory results for WNV or EHV-1; TS2, outbreaks removed based on historical data; TS3, extreme values above the 95% confidence interval deleted.

Fig. 2.

European West Nile virus outbreaks and nervous signs in horses. Number of confirmed cases per week between the first detected case and the last detected case. Dashed grey line indicates outbreak in Italy, 1998 [30], dotted black line indicates outbreak in France, 2004 [18], solid black line indicates outbreak in France, 2000 [13], dashed black line indicates outbreak in France, 2006 [14], solid grey line indicates Hungarian outbreak, 2008 [31]

Fig. 3.

Four examples of simulated data between 2011 and 2013 with one simulated outbreak inserted in each simulated dataset. Outbreak time periods are identified by dotted lines.

Fig. 4.

Raw data from 2014 to 2015 with West Nile virus outbreak identified with dotted lines.

Fig. 5.

Receiver-operating characteristic (ROC) curves for each pre-processing and forecasting method representing median Se_wk (sensitivity based on the number of weeks within an epidemic period detected), plotted against median specificity, Sp. Error bars show the 25% and 75% percentile of the point value over 1500 simulated years and 500 simulated outbreaks. Blue point shows RESPE's current performances. GLM, Generalized linear model; HW, Holt–Winters.

Fig. 6.

Receiver-operating characteristic (ROC) curves for each pre-processing and forecasting method representing median Se_out (sensitivity based on the number of outbreaks detected out of all inserted outbreaks), plotted against median specificity, Sp. Error bars show the 25% and 75% percentile of the point value over 1500 simulated years and 500 simulated outbreaks. HW, Holt–Winters; GLM, generalized linear model.

Fig. 7.

Activity monitoring operation curves for each pre-processing and forecasting method representing median time for outbreak detection, plotted against number of false-positive alarms per year. Error bars show the 25% and 75% percentile of the point value over 1500 simulated years and 500 simulated outbreaks. Blue point indicates RESPE's current performance. HW, Holt–Winters; GLM, generalized linear model.

Fig. 8.

Free-response ROC curves for each pre-processing and forecasting method representing the percentage of outbreaks detected, plotted against the number of false-positive alarms per year. Error bars show the 25% and 75% percentile of the point value over 1500 simulated years and 500 simulated outbreaks. Blue point indicates RESPE's current performance. HW, Holt–Winters; GLM, generalized linear model.

Fig. 9.

Curves for each pre-processing and forecasting method representing the Se_out (sensitivity based on the number of outbreaks detected out of all inserted outbreaks), and specificity, Sp, plotted against the alarm threshold k used for outbreak detection. GLM, Generalized linear model; HW, Holt–Winters.