Visbrain: A Multi-Purpose GPU-Accelerated Open-Source Suite for Multimodal Brain Data Visualization

Abstract

We present Visbrain, a Python open-source package that offers a comprehensive visualization suite for neuroimaging and electrophysiological brain data. Visbrain consists of two levels of abstraction: (1) objects which represent highly configurable neuro-oriented visual primitives (3D brain, sources connectivity, etc.) and (2) graphical user interfaces for higher level interactions. The object level offers flexible and modular tools to produce and automate the production of figures using an approach similar to that of Matplotlib with subplots. The second level visually connects these objects by controlling properties and interactions through graphical interfaces. The current release of Visbrain (version 0.4.2) contains 14 different objects and three responsive graphical user interfaces, built with PyQt: Signal, for the inspection of time-series and spectral properties, Brain for any type of visualization involving a 3D brain and Sleep for polysomnographic data visualization and sleep analysis. Each module has been developed in tight collaboration with end-users, i.e., primarily neuroscientists and domain experts, who bring their experience to make Visbrain as transparent as possible to the recording modalities (e.g., intracranial EEG, scalp-EEG, MEG, anatomical and functional MRI). Visbrain is developed on top of VisPy, a Python package providing high-performance 2D and 3D visualization by leveraging the computational power of the graphics card. Visbrain is available on Github and comes with a documentation, examples, and datasets (http://visbrain.org).

Keywords: EEG; MEG; OpenGL; brain; neuroscience; open-source; python; visualization.

Figure 1

Architecture of the Visbrain software. The left branch in blue illustrates the three included graphical user interfaces (Signal, Brain, and Sleep and). For advanced users that want to interact programmatically with Visbrain, the right branch in red shows 6 of the 14 implemented objects in Visbrain. These objects are presented in circles to emphasize the fact that each of them is independent. Then, using the scene (SceneObj) these objects can be superimposed or juxtaposed into subplots inside a unique figure. The scene offers a finer grain control over the layout. Note that each subplot is interactive, meaning that rotation, translation and zoom can be applied in real time on each subplot.

Figure 2

Illustration of the main features of the brain object (BrainObj). This object delivers some basic features as the possibility to display a translucent or opaque brain mesh (A) or to pick only one hemisphere (B). Intracranial data can also be projected onto the surface (C) and other recording modalities can also be displayed [fMRI activation (D) and MEG data (E)]. In addition, parcels can also be used (G) and data can be assigned to those parcels (H). All of those subplots use MNI templates included with Visbrain, but the user can also define and save a custom template by defining subsets of vertices and faces (F).

Code Snippet 1

Display the left and right hemispheres into two separate subplots along with random MNI sources/contacts/electrodes.

Code Snippet 2

Display the left and right thalamus

Figure 3

Illustration of the main features of the region of interest (ROI) object (RoiObj). Visbrain provides several default atlases that can be used to extract the mesh of specific regions (A,B,F). In addition, the source object (SourceObj) can interact with the ROI object. For example, sources' activity can be projected onto the mesh (C). The RoiObj can also be used to identify in which region a source is contained. Here, sources are color-coded according to the MIST (D) but a table with all of the anatomical informations can also be exported. Finally, it is also possible to keep only the sources that fall into the volume formed by the mesh (E).

Figure 4

Illustration of the main features of the source object (SourceObj) using an intracranial dataset. Additional data can be assigned to sources and the color can either be individually defined or based on a colormap (A). A text can also be attached to sources (B). In a similar way to Figure 3D, here, sources are colored according to Brodmann areas (C). The data attached to sources in (A) is then projected onto the surface of the brain (D) or onto the surface of the default mode network (DMN) (E). Finally, the cortical repartition (F) is the number of contributing sources per vertex. It can be an interesting feature to estimate the number of sources that have contributed to each point of the cortical mesh when projecting source's data.

Figure 5

Illustration of the main features of the connectivity object (ConnectObj). The three sub-visuals express three coloring methods. The first method (A) is simply to color edges by connectivity strength. The second (B) color edges according to the number of connections per node and finally, (C) use colors that are manually defined.

Figure 6

Illustration of additional implemented objects. (A) cross-section of fMRI data (CrossSecObj). The cross-section can be used to load background anatomical images and superimposed activations. It is also possible to move around the volume by clicking on it (still under development). Subplots (B, C) respectively illustrate time-series (TimeSeries3DObj) and pictures (Picture3DObj) embedded inside the mesh. Here, the pictures are time-frequency maps. (D) Plot vector-valued (VectorObj) MEG inverse solution. Visbrain also contains objects to plot images (ImageObj) as illustrated in (E) with a connectivity matrix, time-frequency maps (TimeFrequencyObj) (F), phase-amplitude coupling (PacmapObj) (G). Finally, the TopoObj can be used to plot topographic representations of EEG data, draw levels and connectivity links between EEG sensors (H).

Figure 7

Example of the GUI of the Signal module. Leftmost is the setting panel, and side-by-side are all of the time-series re-arranged into a 2D clickable grid and rightmost, an enlarged version of one of those time-series.

Figure 8

Plotting capabilities of Signal. (A) 104 intracranial recording sites of 4,000 time points each are rearranged into a clickable 13 rows by 8 columns grid. A double-click on one signal of the grid enlarges it in the second layout. This enlarged time-series can either be displayed as a continuous line (B) or a cloud of points (C). A histogram can also be computed (D) as well as the time-frequency map using Morlet's wavelets (E) or the power spectrum density (F).

Figure 9

Thresholding and annotation example of an intracranial time-series. The two horizontal lines indicate the threshold values and time points that are either above or under are turned in red. The green markers show inserted annotations that can also be exported.

Figure 10

Example of the GUI of the Brain module. The settings panel on the left can either be displayed or hidden. This panel contains two tabs: Objects, in order to control the properties of each visual class (e.g., BrainObj, SourceObj, etc.) passed to the interface and Cbar for controlling the colorbar and color properties of a selected object. On the right, the main canvas contains the MNI brain with sources and connectivity links between those sources. This canvas allows fluid rotation, zoom and translation, but also mesh slicing along the (x, y, z) axes. Here, the colorbar of connectivity strength is displayed but it can also be hidden.

Figure 11

Colorbar control. The Cbar tab of the settings panel contains all of the properties to design the colorbar of a specific object (width, colormap, limits, lower and upper thresholds, title, etc.). In addition, these properties can be modified for each object. Here, the widget controls the colorbar for the data projected onto the surface of the brain.