Predicting epidemics using search engine data: a comparative study on measles in the largest countries of Europe

Abstract

Background: In recent years new forms of syndromic surveillance that use data from the Internet have been proposed. These have been developed to assist the early prediction of epidemics in various cases and diseases. It has been found that these systems are accurate in monitoring and predicting outbreaks before these are observed in population and, therefore, they can be used as a complement to other methods. In this research, our aim is to examine a highly infectious disease, measles, as there is no extensive literature on forecasting measles using Internet data, METHODS: This research has been conducted with official data on measles for 5 years (2013-2018) from the competent authority of the European Union (European Center of Disease and Prevention - ECDC) and data obtained from Google Trends by using scripts coded in Python. We compared regression models forecasting the development of measles in the five countries.

Results: Results show that measles can be estimated and predicted through Google Trends in terms of time, volume and the overall spread. The combined results reveal a strong relationship of measles cases with the predicted cases (correlation coefficient R= 0.779 in two-tailed significance p< 0.01). The mean standard error was relatively low 45.2 (12.19%) for the combined results. However, major differences and deviations were observed for countries with a relatively low impact of measles, such as the United Kingdom and Spain. For these countries, alternative models were tested in an attempt to improve the results.

Conclusions: The estimation of measles cases from Google Trends produces acceptable results and can help predict outbreaks in a robust and sound manner, at least 2 months in advance. Python scripts can be used individually or within the framework of an integrated Internet surveillance system for tracking epidemics as the one addressed here.

Keywords: Computational science; Forecasting; Linear regression; Measles; Programming languages; Syndromic surveillance.

Fig. 1

Combined results graph for Italy, Germany and France

Fig. 2

Combined results graph for all countries

Fig. 3

Real cases, Predictions and normal P-P Plots for each country

Fig. 4

Scatterplots for residuals against predicted values

Fig. 5

Histograms for residuals against normal distribution

Fig. 6

Normal Q-Q Plots for residuals

Fig. 7

Standard residuals versus order

Fig. 8

ARIMA model for the UK and Spain