ImageJ2: ImageJ for the next generation of scientific image data

Abstract

Background: ImageJ is an image analysis program extensively used in the biological sciences and beyond. Due to its ease of use, recordable macro language, and extensible plug-in architecture, ImageJ enjoys contributions from non-programmers, amateur programmers, and professional developers alike. Enabling such a diversity of contributors has resulted in a large community that spans the biological and physical sciences. However, a rapidly growing user base, diverging plugin suites, and technical limitations have revealed a clear need for a concerted software engineering effort to support emerging imaging paradigms, to ensure the software's ability to handle the requirements of modern science.

Results: We rewrote the entire ImageJ codebase, engineering a redesigned plugin mechanism intended to facilitate extensibility at every level, with the goal of creating a more powerful tool that continues to serve the existing community while addressing a wider range of scientific requirements. This next-generation ImageJ, called "ImageJ2" in places where the distinction matters, provides a host of new functionality. It separates concerns, fully decoupling the data model from the user interface. It emphasizes integration with external applications to maximize interoperability. Its robust new plugin framework allows everything from image formats, to scripting languages, to visualization to be extended by the community. The redesigned data model supports arbitrarily large, N-dimensional datasets, which are increasingly common in modern image acquisition. Despite the scope of these changes, backwards compatibility is maintained such that this new functionality can be seamlessly integrated with the classic ImageJ interface, allowing users and developers to migrate to these new methods at their own pace.

Conclusions: Scientific imaging benefits from open-source programs that advance new method development and deployment to a diverse audience. ImageJ has continuously evolved with this idea in mind; however, new and emerging scientific requirements have posed corresponding challenges for ImageJ's development. The described improvements provide a framework engineered for flexibility, intended to support these requirements as well as accommodate future needs. Future efforts will focus on implementing new algorithms in this framework and expanding collaborations with other popular scientific software suites.

Keywords: Extensibility; Image processing; ImageJ; ImageJ2; Interoperability; N-dimensional; Open development; Open source; Reproducibility.

Fig. 1

Examples of image processing algorithms available in ImageJ Ops. Panel a (top left): 3D wireframe mesh of ImageJ’s Bat Cochlea Volume sample dataset [94], computed by the geom.marchingCubes op, an implementation of the marching cubes algorithm [95], visualized using MeshLab [96]. Credit to Kyle Harrington for the figure, Tim-Oliver Buchholz for authoring the op, and Art Keating for the dataset. Panel b (top right): Richardson-Lucy Total Variation deconvolution [97] of the Stellaris FISH dataset #1 [98], computed by the deconvolve.richardsonLucyTV op. Credit to Brian Northan for authoring the op and figure [99], and George McNamara for the dataset. Panel c (bottom): Grayscale morphology and neighborhood filter operations on Fiji’s New Lenna sample image, using a diamond-shaped structuring element with radius 3. Credit to Jean-Yves Tinevez, Jonathan Hale and Leon Yang for authoring the ops

Fig. 2

ImageJ update sites provide additional functionality to ImageJ. The Morphological Segmentation plugin, part of the MorphoLibJ plugin collection [100], easily segments the rings of ImageJ’s Tree Rings sample dataset (panel a). The MorphoLibJ plugins are installed into the Fiji distribution of ImageJ by enabling the IJPB-plugins update site (panel b). Credit to David Legland and Ignacio Arganda-Carreras for authoring the plugins

Fig. 3

ImageJ 1.x case logic compared to a unified ImgLib2 implementation. Panel a (left) shows the ImageJ 1.x implementation of a rolling ball background subtraction method, part of the ij.plugin.filter.BackgroundSubtracter class. Panel b (right) shows an equivalent implementation using ImgLib2, without the need for extensive case logic

Fig. 4

Side-by-side comparison of ImageJ2-based user interfaces and integrations. Panel a (top left): ImageJFX, a JavaFX-based user interface built on ImageJ2. Panel b (top right): ImageJ2’s default user interface, the ImageJ Legacy UI, which wraps ImageJ 1.x. Panel c (bottom left): Example KNIME workflow utilizing ImageJ2 image processing nodes. Panel d (middle right): Swing UI prototype, closely modeled after ImageJ 1.x so that it remains familiar to existing users, in various Java “Look & Feel” modes. Panel e (bottom right): A proof-of-concept Apache Pivot user interface. The ImageJFX and ImageJ Legacy UIs display an XY slice of ImageJ’s Confocal Series sample dataset (dataset courtesy of Joel Sheffield), which has been rotated, smoothed and colorized

Fig. 5

Comparison of pure ImageJ 1.x command with one using SciJava declarative syntax. Panel a (left) shows an ImageJ 1.x implementation of a plugin that copies slice labels from one image to another, as chosen by the user. Panel b (right) shows the same plugin written using the SciJava declarative command syntax. Changed lines are highlighted in blue, new lines in green. The actual operation (the copyLabels method) is identical, but the routine for selecting which images to process is no longer necessary

Fig. 6

Benchmarks of a simple addition operation with ImageJ Ops and ImageJ 1.x. Time performance comparison of simple addition operations between raw Java array manipulation, various math.add operations of ImageJ Ops, and ImageJ 1.x’s Process ⊳ Math ⊳ Add… command. Benchmarks were run for 20 rounds on randomly generated uint8 noise images dimensioned 15,000×15,000, using the JUnit Benchmarks framework, on a MacBook Pro (Retina, 15-inch, Mid 2015) running macOS Sierra 10.12 with 2.5 GHz Intel Core i7 processor and 16 GB 1600 MHz DDR3 memory. Positive numbers are fold faster, negative numbers are fold slower. The routines which produced these results can be found in the ImageJ Ops test code, in the AddOpBenchmarkTest class of the net.imagej.ops.benchmark package

Fig. 7

A mixed-world ImageJ 1.x + ImageJ2 script. This example Python script (panel d) uses ImageJ Ops to preprocess a confocal image and perform an automatic thresholding (panel b). ImageJ 1.x’s Analyze Particles routine is then called to isolate (panel a) and measure (panel c) foreground objects. Script contributed by Brian Northan, True North Intelligent Algorithms LLC. This script is available within ImageJ as a sample from the Tutorials submenu of the Script Editor’s Templates menu