Web tools for large-scale 3D biological images and atlases

Abstract

Background: Large-scale volumetric biomedical image data of three or more dimensions are a significant challenge for distributed browsing and visualisation. Many images now exceed 10GB which for most users is too large to handle in terms of computer RAM and network bandwidth. This is aggravated when users need to access tens or hundreds of such images from an archive. Here we solve the problem for 2D section views through archive data delivering compressed tiled images enabling users to browse through very-large volume data in the context of a standard web-browser. The system provides an interactive visualisation for grey-level and colour 3D images including multiple image layers and spatial-data overlay.

Results: The standard Internet Imaging Protocol (IIP) has been extended to enable arbitrary 2D sectioning of 3D data as well a multi-layered images and indexed overlays. The extended protocol is termed IIP3D and we have implemented a matching server to deliver the protocol and a series of Ajax/Javascript client codes that will run in an Internet browser. We have tested the server software on a low-cost linux-based server for image volumes up to 135GB and 64 simultaneous users. The section views are delivered with response times independent of scale and orientation. The exemplar client provided multi-layer image views with user-controlled colour-filtering and overlays.

Conclusions: Interactive browsing of arbitrary sections through large biomedical-image volumes is made possible by use of an extended internet protocol and efficient server-based image tiling. The tools open the possibility of enabling fast access to large image archives without the requirement of whole image download and client computers with very large memory configurations. The system was demonstrated using a range of medical and biomedical image data extending up to 135GB for a single image volume.

Figure 1

Sectioning plane definition. The viewing plans plane is at distance d from the fixed point f and perpendicular to the direction defined by angles pitch and yaw. The angle roll defines the orientation of section image when projected onto the view window which in our case is the user’s screen.

Figure 2

Example controls for the IIP3D viewer: a - image locator, the blue rectangle indicates the currently visible region and can be dragged to any required location within the full image; b & c - image selection, the red line is dragged to select a new section image; d - 3D section controls with the angle feedback at the top and specific sliders for pitch, yaw, roll, scale and distance.e - layer controls selecting visibility, opacity and if needed a colour filter to distinguish multiple grey-level image overlays; f - graphical overlay controls in this case for anatomy regions with the option to control visibility and colour.

Figure 3

IIP3D Viewer design is based on the MVC design pattern: (a) Sequence of a web page request is server by a sequence of php and JavaScript pages that implement the MVC. Asynchronous tile image load is provided by AJAX; (b) Controllers provides control through the UI of Mootols library. The model encodes current viewing parameters set by the controllers and visualised by the tileImageView.

Figure 4

Architecture of IIP3D server using a proxy server. The web server passes the user requests to the proxy, which forwards them to individual IIP servers. These servers have direct access to the Woolz Object and return the requested data. The numbered lines show the order of the requests (continuous lines) and the replies (dotted lines).

Figure 5

An section image consisting of 2 × 3 tiles. The tiles are obtained by 6 requests that are identical except the required tile number of the command.

Figure 6

Multi channel request withSEL. The compound Woolz object consists of several channels with multiple domains, each of which is sectioned individually. They may be queried separately or combined into a single layer in sequential order of the command. The colour and transparency values for each layer are selected independently.

Figure 7

Performance evaluation for four test cases: the direct sets replicate browsing pattern manually performed by a user over 273 seconds; the randomised requests for each client add a random [0.0; 1.0] value to the requested yaw angle, therefore it forces recomputing the sections and does not use cache. Both direct and randomised have a wait (W) and no wait (NW) version, which either respect the timing of the manual requests (e.g. waits between consecutive requests) or it does requests tile one after the other. a) Average request time per number of concurrent clients, where dots show the individual request times for each client and b) Average running time for a test set on clients.

Figure 8

Browsing 3D data using the IIP3D web-browser interface. (a) view of the eMouseAtlas.org E14.5 mouse embryo model. The left-hand side panel provides slider controls to adjust the sectioning orientation which is indicated by the 3D rendering. In the main area the virtual section is displayed with the context menu which indicates the modes available. In this view the measurement mode has been selected and the two cyan crosses indicate the transverse distance across this view of the liver. (b) Waxholm space P56 mouse brain. In this view two of the possible 5 modalities have been selected and displayed as two layers. The T2* MR image in grey is overlaid with the histology reconstruction shown as a magenta overlay set to be partially transparent.

Figure 9

Example of a large image object. Visible Male 140GB dataset, with tools anchored to the left area and viewing regions at the right. This figure also illustrates that the IIP3D server can provide section views through colour volumetric images.

Figure 10

Layered browsing. (a) shows a sagittal view through the HUDSEN CS17 human embryo with anatomy delineation and in situ gene-expression overlay. Pausing the cursor over a painted region will display the name of the underlying region and mouse double-click will result in selection of the matching term in the ontology display tool. (b) shows the virtual flybrain interface which has been further developed to enable query of anatomy term definitions and associated gene-expression and connectivity databases.