Svetlana a supervised segmentation classifier for Napari

Abstract

We present Svetlana (SuperVised sEgmenTation cLAssifier for NapAri), an open-source Napari plugin dedicated to the manual or automatic classification of segmentation results. A few recent software tools have made it possible to automatically segment complex 2D and 3D objects such as cells in biology with unrivaled performance. However, the subsequent analysis of the results is oftentimes inaccessible to non-specialists. The Svetlana plugin aims at going one step further, by allowing end-users to label the segmented objects and to pick, train and run arbitrary neural network classifiers. The resulting network can then be used for the quantitative analysis of biophysical phenoma. We showcase its performance through challenging problems in 2D and 3D and provide a comprehensive discussion on its strengths and limits.

Keywords: Biomedical imaging; Classification; Convolutional neural networks; Efficient AI; Image analysis; Microscopy; Segmentation; Software.

Figure 1

A schematic overview of the Svetlana plugin. (a) A screenshot of the plugin in action. In this example, Svetlana is able to separate the mesorderm (in green) and the neural tube (in red) nuclei of a quail embryo with a high accuracy after just a few clicks. (b) Overview of the Svetlana’s three-step pipeline: Annotation, Training and Prediction. Given pairs of images and segmentation masks, the user labels a few connected components. This set is then used to train a neural network classifier. Once trained, it can be used to classify one or multiple segmented images. (c) Svetlana offers many neural network architectures with increasing complexity. The minimalist architectures can be trained faster and are usually enough to lead to high classification accuracies. (d) The training can be enriched with a large variety of image augmentation techniques available in Albumentations. (e) Online resources are available to assist the user. (f) All experiments can be fine tuned and reproduced using a JSON configuration file.

Figure 2

A histopathology images classification problem. (a) A subset of 6 images taken from PanNuke dataset. (b) and (c) Example of $137\times 137$ pixels training patches showing neoplastic and inflammatory cells respectively. The nucleus of interest has been circled in green for visualization. We can see that the two classes are difficult to distinguish visually for a non-expert. (d) Classification performance by training the model on increasingly large 2-class (neoplastic/inflammatory cells) small datasets. (e) Classification performance on a 5-class problem training with a large dataset.

Figure 3

The osteoclasts classification—(a,b) An example of an entire $8000\times 8000$ pixels image and a classification result using Svetlana. The green spots correspond to activated osteoclasts. The image was provided by . Atlantic Bone Screen company (ABS) (c) $2000\times 2000$ pixels crops of four different images illustrating the diversity of the dataset. (d,e) $750\times 750$ pixels crop and its classification result after training a neural network with 600 annotations (out of 16,671). (f) Example of activated osteoclats. (g) Example of non-activated osteoclasts. (h) This graph shows the excellent correlation (96% coefficient of determination) between Svetlana’s counting and a human counting. Various conditions were tested. For the red dots, no drug was applied. For the green diamonds, drugs inducing cell proliferation were tested. For the purple triangles, drugs inducing cell death were introduced.

Figure 4

Comparing random forests and neural networks—(a) The principle of random forest classifiers: a few (10–1000) random decision trees are constructed and a majority voting allows making a prediction. (b) A convolutional neural network classifier makes a prediction by a sequence of convolutions, nonlinear activation functions and pooling. (c–h) Various classification tasks performed by each classifier. In order to facilitate the visualization, only small portions of the images are displayed. (c,d) Synthetic cell classification with differences of average gray level. Both Ilastik and a simple neural network yields 100% accuracy with as little as 10 labels. (e–f) Anisotropic texture classification. The two textures possess the same mean and variance, but different orientations. The random forest yields unsatisfactory results (64%) whatever the number of labels. The neural network yields 100% accuracy with only 45 labels. The reason for the failure of the random forest is that no pre-defined feature allows discriminating the texture orientations. On their side, neural networks are able to learn the right features with just a few annotations. (g–h) 2D slice of a two-photon microscope image of a quail embryo (courtesy of B. Bénazéraf). It shows a neural tube surrounded by somites. (g) From left to right: the slice containing 854 nuclei—the classification result using 35 labels. Thanks to the use of spatialization parameters, Ilastik provides excellent results for this task. However, if the classifier is applied to the rotated image, it yields unsatisfactory results since the spatialization changed. If the spatialization features are removed to construct the decision trees, Ilastik fails to classify the cells. (h) Svetlana result with 169 labels: it leads to a few mis-classifications on the original image. Nevertheless, it remains effective when the image is rotated thanks to the use of data augmentation during the training. This experiment overall shows the advantages of learning the classification features and to use data augmentation to add desirable properties such as rotation invariance.