Immersive and interactive visualization of 3D spatio-temporal data using a space time hypercube: Application to cell division and morphogenesis analysis

Abstract

The analysis of multidimensional time-varying datasets faces challenges, notably regarding the representation of the data and the visualization of temporal variations. We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data. Additionally, we propose a set of tools allowing the user to filter and manipulate the 3D STC which benefits the visualization, exploration and interaction possibilities offered by VR. Finally, we evaluated the proposed visualization method in the context of 3D temporal cell imaging data analysis, through a user study (n = 5) reporting the feedback from five biologists. These domain experts also accompanied the application design as consultants, providing insights on how the STC visualization could be used for the exploration of complex 3D temporal morphogenesis data.

Keywords: dimension reduction; immersive analytics; interaction; space-time cube; spatio-temporal visualization; virtual reality.

FIGURE 1

From a cross-section on the 3D surface-based temporal data shown on the first figure, we generate a Space-Time Cube visualization, displayed in the second image, showing the evolution over time of the spatial data of the cross-section displayed on the x and y axes. The third picture shows how the visualization can be enriched with numerical and categorical data using different color coding. A set of interaction tools help the user to explore the generated visualization, as seen in the last image.

FIGURE 2

Flow diagram of the STC generation. In step 1), the user places the interactive clipping plane to get the desired cross-section. In step 2), camera parameters are automatically set in order to render the cross section at each time point. The image presents the output of the rendering operation, using the RGB channels to save cell identifiers. Stacking the rendered images yields a 3d volume, as shown in step 3). Each voxel contains a cell identifier, and its position in terms of depth indicates a time point t _i .

FIGURE 3

STC (right) and meshed model (left) visualizations in the VR framework. On the laser selector, a list of available information allowed the user to color map the remaining lifespan of cells on the visualizations. A cell is selected on the meshed model. The name of the cell is displayed on the pointer, and feedback appears on the STC, highlighting the lineage of the cell.

FIGURE 4

The STC at the left presents the cavity formed by gastrulation. The cross-section at the right shows the deformation of cells involved in this particular morphogenetic movement.

FIGURE 5

Cross-section and related STC generated. The color displayed on the STC corresponds to the volume of cells, and helps distinguish 4 parts.

FIGURE 6

Cross-section of the STC. Two divisions are indicated. The membrane of the original cells and their respective daughters cells are highlighted in red.

FIGURE 7

Examples of STCs based on 3 different base planes for the capture, on each example dataset presented in Section 5. The left side of the image is extracted from the shoot-apical meristem dataset, colormapped with numerical values of stress magnitude. The images on the right side are extracted from the C. elegans embryo dataset. Each color represents the successive divisions of the cells present at t = t ₁.