Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 Jan 23:13:12.
doi: 10.1186/1472-6947-13-12.

in silico surveillance: evaluating outbreak detection with simulation models

Bryan Lewis  1 , Stephen Eubank, Allyson M Abrams, Ken Kleinman
Affiliations

in silico surveillance: evaluating outbreak detection with simulation models

Bryan Lewis et al. BMC Med Inform Decis Mak. .

Abstract

Background: Detecting outbreaks is a crucial task for public health officials, yet gaps remain in the systematic evaluation of outbreak detection protocols. The authors' objectives were to design, implement, and test a flexible methodology for generating detailed synthetic surveillance data that provides realistic geographical and temporal clustering of cases and use to evaluate outbreak detection protocols.

Methods: A detailed representation of the Boston area was constructed, based on data about individuals, locations, and activity patterns. Influenza-like illness (ILI) transmission was simulated, producing 100 years of in silico ILI data. Six different surveillance systems were designed and developed using gathered cases from the simulated disease data. Performance was measured by inserting test outbreaks into the surveillance streams and analyzing the likelihood and timeliness of detection.

Results: Detection of outbreaks varied from 21% to 95%. Increased coverage did not linearly improve detection probability for all surveillance systems. Relaxing the decision threshold for signaling outbreaks greatly increased false-positives, improved outbreak detection slightly, and led to earlier outbreak detection.

Conclusions: Geographical distribution can be more important than coverage level. Detailed simulations of infectious disease transmission can be configured to represent nearly any conceivable scenario. They are a powerful tool for evaluating the performance of surveillance systems and methods used for outbreak detection.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Finite state machine representation allows for a flexible representation of a disease process. Each state determines the duration in that state, level of symptoms, susceptibility, and infectiousness. For example, susceptible individuals have a 75% chance that upon exposure to an infectious contact and successful infection they will transition to the latent Exposed state in ILI Manifestation 1; after 1 to 2 days they will then transition to one of the more Infectious states with a 50% chance of having symptoms.
Figure 2
Figure 2
Two years of real ILI data (red) compared to 2 years of data from a single simulation (black). Each dot represents the total number of cases seen in the surveillance stream on a single day; the curves are a smoothed fit to these data.
Figure 3
Figure 3
Simulated ILI surveillance data for downtown Boston as captured by the “Natural 6” surveillance system. Surveillance counts per day centered in each zip code location are shown as histograms within each zip code. Detection of an inserted test outbreak (red triangle) is indicated by red bordered zip codes and a false-positive outbreak by black bordered zip codes.
Figure 4
Figure 4
Pseudo-ROC curves of outbreak detection. Proportion detected for each surveillance system vs. proportion of all false-positives identified.
Figure 5
Figure 5
Proportion detected vs. mean days to detection across decision thresholds.
See this image and copyright information in PMC

References

    1. Buehler JW, Hopkins RS, Overhage JM, Sosin DM, Tong V. Group CW. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC working group. MMWR Recomm Rep. 2004;53(RR-5):1–11. - PubMed
    1. Buckeridge DL, Switzer P, Owens D, Siegrist D, Pavlin J, Musen M. An evaluation model for syndromic surveillance: assessing the performance of a temporal algorithm. MMWR Morb Mortal Wkly Rep. 2005;54(Suppl):109–115. - PubMed
    1. Kleinman KP, Abrams A, Mandl K, Platt R. Simulation for assessing statistical methods of biologic terrorism surveillance. MMWR Morb Mortal Wkly Rep. 2005;54(Suppl):101–108. - PubMed
    1. Eubank S, Guclu H, Kumar VSA, Marathe MV, Srinivasan A, Toroczkai Z, Wang N. Modelling disease outbreaks in realistic urban social networks. Nature. 2004;429(6988):180–184. doi: 10.1038/nature02541. - DOI - PubMed
    1. Halloran ME, Ferguson NM, Eubank S, Longini IM, Cummings DAT, Lewis B, Xu S, Fraser C, Vullikanti A, Germann TC. et al.Modeling targeted layered containment of an influenza pandemic in the United States. Proc Natl Acad Sci USA. 2008;105(12):4639–4644. doi: 10.1073/pnas.0706849105. - DOI - PMC - PubMed

Publication types

LinkOut - more resources