Using temporal context to improve biosurveillance

Abstract

Current efforts to detect covert bioterrorist attacks from increases in hospital visit rates are plagued by the unpredictable nature of these rates. Although many current systems evaluate hospital visit data 1 day at a time, we investigate evaluating multiple days at once to lessen the effects of this unpredictability and to improve both the timeliness and sensitivity of detection. To test this approach, we introduce simulated disease outbreaks of varying shapes, magnitudes, and durations into 10 years of historical daily visit data from a major tertiary-care metropolitan teaching hospital. We then investigate the effectiveness of using multiday temporal filters for detecting these simulated outbreaks within the noisy environment of the historical visit data. Our results show that compared with the standard 1-day approach, the multiday detection approach significantly increases detection sensitivity and decreases latency while maintaining a high specificity. We conclude that current biosurveillance systems should incorporate a wider temporal context to improve their effectiveness. Furthermore, for increased robustness and performance, hybrid systems should be developed to capitalize on the complementary strengths of different types of temporal filters.

Figure 1

The distribution of forecast errors (in visits per day) from a historical model of emergency department visit rates. These errors inhibit reliable detection of outbreaks. Specifically, minor outbreaks that cause only small increases in visit rates can be totally masked by these forecast errors.

Figure 2

The shapes of four multiday temporal filters used for detecting disease outbreaks with a 7-day detection time window. The value for each day represents the relative weight attributed to that day by the detection filter.

Figure 3

Simulation results. (Upper) Stimulus: When adding simulated outbreaks (7-day, flat, size 20) to the noisy historical visit data, some outbreaks (red) are masked by the noise, appearing broken up in the resultant input exceedance signal (blue; actual visits minus expected visits plus outbreaks). (Lower) Response: The responses of the different temporal filters to the stimulus above. The dashed lines are the alarm thresholds for the various filters. Some filters react quickly to increased visit rates, whereas others react more slowly.

Figure 4

ROC curve shows the tradeoff between sensitivity and specificity for all four filters, given outbreaks of size 20 visits per day.

Figure 5

Sensitivities using the benchmark specificity of 0.97 (Upper) and areas under the ROC curve (Lower) of the four filters, shown for a range of outbreak sizes.

Figure 6

Timeliness of detection: Sensitivities of all four filters during different stages of the outbreaks, using the benchmark specificity of 0.97. Comparisons are shown for different outbreak sizes 30 (Top), 20 (Middle), and 10 (Bottom).