Assessing Google flu trends performance in the United States during the 2009 influenza virus A (H1N1) pandemic

Abstract

Background: Google Flu Trends (GFT) uses anonymized, aggregated internet search activity to provide near-real time estimates of influenza activity. GFT estimates have shown a strong correlation with official influenza surveillance data. The 2009 influenza virus A (H1N1) pandemic [pH1N1] provided the first opportunity to evaluate GFT during a non-seasonal influenza outbreak. In September 2009, an updated United States GFT model was developed using data from the beginning of pH1N1.

Methodology/principal findings: We evaluated the accuracy of each U.S. GFT model by comparing weekly estimates of ILI (influenza-like illness) activity with the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet). For each GFT model we calculated the correlation and RMSE (root mean square error) between model estimates and ILINet for four time periods: pre-H1N1, Summer H1N1, Winter H1N1, and H1N1 overall (Mar 2009-Dec 2009). We also compared the number of queries, query volume, and types of queries (e.g., influenza symptoms, influenza complications) in each model. Both models' estimates were highly correlated with ILINet pre-H1N1 and over the entire surveillance period, although the original model underestimated the magnitude of ILI activity during pH1N1. The updated model was more correlated with ILINet than the original model during Summer H1N1 (r = 0.95 and 0.29, respectively). The updated model included more search query terms than the original model, with more queries directly related to influenza infection, whereas the original model contained more queries related to influenza complications.

Conclusions: Internet search behavior changed during pH1N1, particularly in the categories "influenza complications" and "term for influenza." The complications associated with pH1N1, the fact that pH1N1 began in the summer rather than winter, and changes in health-seeking behavior each may have played a part. Both GFT models performed well prior to and during pH1N1, although the updated model performed better during pH1N1, especially during the summer months.

Figure 1. Time series plots of ILINet data and original and updated GFT estimates.

A) ILINet data and GFT estimates from 2009. B) ILINet data and GFT estimates for the entire time period where GFT estimates are available: 2003–2009.

Figure 2. Time series plots of ILINet data and category-level GFT estimates.

Category-level estimates are created by applying the GFT methodology to a subset of the queries in a given model. A) ILINet data and GFT estimates based on original model queries related to influenza complications. B) ILINet data and GFT estimates based on updated model queries related to specific influenza symptoms.

Figure 3. Time series plots of ILINet data and query-level GFT estimates.

Query-level estimates are created by applying the GFT methodology to the search activity for a single query. A) ILINet data and GFT estimates based on the query [symptoms of flu]. B) ILINet data and GFT estimates based on the query [symptoms of bronchitis]. C) ILINet data and GFT estimates based on the query [symptoms of pneumonia].