Run-time interoperability between neuronal network simulators based on the MUSIC framework

Abstract

MUSIC is a standard API allowing large scale neuron simulators to exchange data within a parallel computer during runtime. A pilot implementation of this API has been released as open source. We provide experiences from the implementation of MUSIC interfaces for two neuronal network simulators of different kinds, NEST and MOOSE. A multi-simulation of a cortico-striatal network model involving both simulators is performed, demonstrating how MUSIC can promote inter-operability between models written for different simulators and how these can be re-used to build a larger model system. Benchmarks show that the MUSIC pilot implementation provides efficient data transfer in a cluster computer with good scaling. We conclude that MUSIC fulfills the design goal that it should be simple to adapt existing simulators to use MUSIC. In addition, since the MUSIC API enforces independence of the applications, the multi-simulation could be built from pluggable component modules without adaptation of the components to each other in terms of simulation time-step or topology of connections between the modules.

Fig. 1

Illustration of a typical multi-simulation using MUSIC. Three applications, A, B, and C, are exchanging data during runtime. Each application runs in a set of MPI processes. Data flows exit and enter ports, each spanning the set of processes of the application

Fig. 2

Data transfer over a connection from an application running in four processes to an application running in three processes. The light gray areas in the sender and receiver represent the MUSIC port. Dashed lines divide the application into distinct processes. The width of the port is the total number of distinct data items being communicated from all sender processes to the receiver processes

Fig. 3

The UML diagram shows the data members and the functions of the proxy that represents MUSIC output ports in NEST. The new class is shown in grey

Fig. 4

a Nodes in NEST are distributed over the processes (p = 0,1,2). iaf denotes an integrate and fire neuron, (iaf) denotes a proxy. mop denotes a music_out_proxy. MUSIC channels are indicated in square brackets for each connection (arrows). b A sketch of the complete connectivity from the nodes (lower squares) over the different channels (numbers in square brackets) to MUSIC. The dashed box encloses all proxies that belong to one MUSIC output port

Fig. 5

The UML diagram shows the data members and the functions of the proxy that represents a channel on a MUSIC input port in NEST and its relation to the class that represents the MUSIC input port. New classes are shown in grey

Fig. 6

a Nodes in NEST are distributed over the processes (p = 0,1,2). iaf denotes an integrate and fire neuron, (iaf) denotes a proxy. mip denotes a music_in_proxy. The numbers on the left indicate the global id of the nodes. MUSIC channel ids are indicated in square brackets for each music_in_proxy. The STDP connection is indicated by a dashed arrow. b A sketch of the complete connectivity from MUSIC (channels in square brackets) to the MUSIC event handler (grey rectangles) to the proxies (squares labeled 1 and 2) to the actual target nodes (lower squares). The STDP connection is indicated by a dashed arrow. The dashed box encloses all event handlers and proxies that represent a MUSIC input port

Fig. 7

Illustration of the MOOSE messaging structure. Two compartments are connected by a synapse

Fig. 8

A model in MOOSE receiving spike-time information from MUSIC. An object of type InputEventPort handles spike-times relayed by MUSIC. Objects of type InputEventChannel act as proxies for the spike-generating entities in the foreign application. The proxies forward the spikes to targets in the model via messages. Note that it is possible for a message to connect a proxy and its target even if both are in separate processes. It is most efficient, however, if they are on the same process

Fig. 9

Benchmark models. a Asymmetric benchmark model consisting of one large-scale cortex model and a single process relay model. Inter-model communication via MUSIC is bidirectional but asymmetric, mainly from the cortex model to the relay model. b Symmetric benchmark model consisting of two interconnected large-scale cortex models. Communication via MUSIC is symmetric between the two models

Fig. 10

Performance of the layered cortical network model and the asymmetric multi-simulation benchmark. a Computing time per second of biological time as a function of the number of compute cores. Gray diamonds show the performance of the cortex model simulation when NEST is compiled without MUSIC, black squares the performance when compiled with MUSIC. The dotted line indicates the expectation for linear speed-up. b Computing time per second of biological time of the asymmetric multi-simulation benchmark. The number of cores corresponds to the number of cores used for the cortex model without the additional core for the relay network. The shown data corresponds to N_MUSIC = 8 (black circles) and N_MUSIC = 71,000 (gray triangles); the dotted line gives the expectation for linear speed-up

Fig. 11

Performance of the symmetric multi-simulation benchmark. a Simulation time per second of biological time as a function of the total number of compute cores for both network models. Black squares show the performance of the control (N_MUSIC = 0), dark gray diamonds the benchmark’s performance for N_MUSIC = 8 and light gray circles for N_MUSIC = 8,000. The dotted line indicates the expectation for linear speed-up of the control. b Simulation time per second of biological time as a function of the MUSIC spike rate. Simulations with 32 cores are indicated in black, with 64 cores in dark gray. Dashed lines indicate the control (N_MUSIC = 0) and squares show the data for the symmetric multi-simulation benchmark with light gray lines showing the corresponding linear fits

Fig. 12

Schematic of run-time interoperability for a cortico-striatal model. The cortical model simulated in NEST uses MUSIC to send spikes to the striatal model in MOOSE. In addition, two visualization processes receive the spike information from both NEST and MOOSE

Fig. 13

Results from the multi-simulation described schematically in Fig. 12. To the left, two window captures from 3D visualizations of the cortex and striatum model are shown. In the upper half of the figure, 500 outputs from the cortex model in NEST are visualized on a planar grid, the radii and intensity of the color of the neurons increase when they spike. In the lower part, 10 MS (red) and 10 FS (blue) neurons in the striatal network are visualized in the same manner. To the right are a raster plot of the cortical activity and voltage traces for the MS and FS neurons