A cloud-based simulation architecture for pandemic influenza simulation

Abstract

High-fidelity simulations of pandemic outbreaks are resource consuming. Cluster-based solutions have been suggested for executing such complex computations. We present a cloud-based simulation architecture that utilizes computing resources both locally available and dynamically rented online. The approach uses the Condor framework for job distribution and management of the Amazon Elastic Computing Cloud (EC2) as well as local resources. The architecture has a web-based user interface that allows users to monitor and control simulation execution. In a benchmark test, the best cost-adjusted performance was recorded for the EC2 H-CPU Medium instance, while a field trial showed that the job configuration had significant influence on the execution time and that the network capacity of the master node could become a bottleneck. We conclude that it is possible to develop a scalable simulation environment that uses cloud-based solutions, while providing an easy-to-use graphical user interface.

Figure 1.

Overview of the PROSPER protocol in relation to infectious disease response program implementation.

Figure 2.

A general description of the scientific workflow involved in pandemic simulations.

Figure 3.

The architecture for the cloud-based solution based on Condor and EC2.

Figure 4.

Section of the web-based panel. It allows users to add more Condor worker nodes by instantiating (booting) new EC2 images. The user can select a machine image to start and can monitor already running virtual machines.

Figure 5.

The Condor job distribution and result collection process, with the data transfer as the potential bottleneck in the process.