Application of change point analysis to daily influenza-like illness emergency department visits

Abstract

Background: The utility of healthcare utilization data from US emergency departments (EDs) for rapid monitoring of changes in influenza-like illness (ILI) activity was highlighted during the recent influenza A (H1N1) pandemic. Monitoring has tended to rely on detection algorithms, such as the Early Aberration Reporting System (EARS), which are limited in their ability to detect subtle changes and identify disease trends.

Objective: To evaluate a complementary approach, change point analysis (CPA), for detecting changes in the incidence of ED visits due to ILI.

Methodology and principal findings: Data collected through the Distribute project (isdsdistribute.org), which aggregates data on ED visits for ILI from over 50 syndromic surveillance systems operated by state or local public health departments were used. The performance was compared of the cumulative sum (CUSUM) CPA method in combination with EARS and the performance of three CPA methods (CUSUM, structural change model and Bayesian) in detecting change points in daily time-series data from four contiguous US states participating in the Distribute network. Simulation data were generated to assess the impact of autocorrelation inherent in these time-series data on CPA performance. The CUSUM CPA method was robust in detecting change points with respect to autocorrelation in time-series data (coverage rates at 90% when -0.2≤ρ≤0.2 and 80% when -0.5≤ρ≤0.5). During the 2008-9 season, 21 change points were detected and ILI trends increased significantly after 12 of these change points and decreased nine times. In the 2009-10 flu season, we detected 11 change points and ILI trends increased significantly after two of these change points and decreased nine times. Using CPA combined with EARS to analyze automatically daily ED-based ILI data, a significant increase was detected of 3% in ILI on April 27, 2009, followed by multiple anomalies in the ensuing days, suggesting the onset of the H1N1 pandemic in the four contiguous states.

Conclusions and significance: As a complementary approach to EARS and other aberration detection methods, the CPA method can be used as a tool to detect subtle changes in time-series data more effectively and determine the moving direction (ie, up, down, or stable) in ILI trends between change points. The combined use of EARS and CPA might greatly improve the accuracy of outbreak detection in syndromic surveillance systems.

Figure 1

Proportion of emergency department visits due to influenza-like illness by age group for the period, October 4, 2008–October 9, 2010, in one U.S. Department of Health and Human Services region.

Figure 2

Proportion of emergency department visits due to influenza-like illness (ILI) for all ages, October 4, 2008 through October 4, 2010 in four states (change points marked in different color representing the trend of ILI activity). Note: moderately up (Δ>1%), slightly up (0<Δ≤1%), slightly down (−1%<Δ≤0), and moderately down (Δ≤−1%), where Δ is the difference in the mean of %ILI between the interval after the change point and the interval before the change point.

Figure 3

Modified Early Aberration Reporting System (EARS) C2 anomalies and Taylor cumulative sum change point analysis (CPA) change points detected in analysis of influenza-like illness emergency department visit data, four states, October 4, 2008–October 9, 2010 (red cross, EARS C2 anomaly; vertical line, CPA change point).

Figure 4

Scatter plot of coverage probability by change point analysis method in the autocorrelation simulated data. Note: cusum1 and scm1 refers to those change points in the autocorrelated data which are exactly the same as those detected from independent and identically distributed (iid) samples, and cusum2 and scm2 allows ±3 time points away from those detected from iid samples. ρ is the autocorrelation coefficient in the simulated data. CUSUM, cumulative sum; SCM, structural change model.