Large-scale modeling of epileptic seizures: scaling properties of two parallel neuronal network simulation algorithms

Abstract

Our limited understanding of the relationship between the behavior of individual neurons and large neuronal networks is an important limitation in current epilepsy research and may be one of the main causes of our inadequate ability to treat it. Addressing this problem directly via experiments is impossibly complex; thus, we have been developing and studying medium-large-scale simulations of detailed neuronal networks to guide us. Flexibility in the connection schemas and a complete description of the cortical tissue seem necessary for this purpose. In this paper we examine some of the basic issues encountered in these multiscale simulations. We have determined the detailed behavior of two such simulators on parallel computer systems. The observed memory and computation-time scaling behavior for a distributed memory implementation were very good over the range studied, both in terms of network sizes (2,000 to 400,000 neurons) and processor pool sizes (1 to 256 processors). Our simulations required between a few megabytes and about 150 gigabytes of RAM and lasted between a few minutes and about a week, well within the capability of most multinode clusters. Therefore, simulations of epileptic seizures on networks with millions of cells should be feasible on current supercomputers.

Figure 1

Example of emergent behavior produced by this type of simulation (using Verdandi). Seizure-like oscillation in a patch of 90 K neurons as displayed by the pseudo-EEG. Below are the action potential trains of ten representative cells of each kind: superficial pyramidal cells (SPYR), deep pyramidal cells (DPYR), large basket cells (BASK), and chandelier cells (CHAN). The ten cells are selected from different locations in the simulated network.

Figure 2

Maximum memory usage per process for pNeo on Beagle as a function of network size on a log-log plot with basis 10. Different lines depict simulations that used a different number of MPI processes. Results on Fusion were essentially identical. The results for Verdandi were qualitatively similar. The total memory used by each simulation can be obtained by multiplying these numbers by the number of MPI processes and it is reasonably constant for each network size.

Figure 3

Strong scaling plots for calculations performed on Beagle: (a) Verdandi and (b) pNeo. Run time is the time it took to complete a simulation, from beginning to end. Most of the measurements exhibit nearly perfect strong scaling.

Figure 4

Increase in total time (sum of the time taken by each thread in a simulation and called “Parallelization penalty” because it is the total CPU time allocated to the simulation) as the number of OpenMP threads is increased while keeping the number of MPI processes at 1 (single node). This calculation was performed on Beagle.

Figure 5

Run time (wall clock time here, not total time as in the previous plot) for a 100 K-cell network on 9 nodes (216 cores) of Beagle. The number of MPI processes was varied while the number of OpenMP threads was adjusted so that each run used all 216 available cores. The leftmost point corresponds to 3 MPI processes per node with 8 OpenMP threads per process.