Generic Isolated Cell Image Generator

Abstract

Building automated cancer screening systems based on image analysis is currently a hot topic in computer vision and medical imaging community. One of the biggest challenges of such systems, especially those using state-of-the-art deep learning techniques, is that they usually require a large amount of training data to be accurate. However, in the medical field, the confidentiality of the data and the need for medical expertise to label them significantly reduce the amount of training data available. A common practice to overcome this problem is to apply data set augmentation techniques to artificially increase the size of the training data set. Classical data set augmentation methods such as geometrical or color transformations are efficient but still produce a limited amount of new data. Hence, there has been interest in data set augmentation methods using generative models able to synthesize a wider variety of new data. VitaDX is actually developing an automated bladder cancer screening system based on the analysis of cell images contained in urinary cytology digital slides. Currently, the number of available labeled cell images is limited and therefore exploitation of the full potential of deep learning techniques is not possible. In an attempt to increase the number of labeled cell images, a new generic generator for 2D cell images has been developed and is described in this article. This framework combines previous works on cell image generation and a recent style transfer method referred to as doodle-style transfer in this article. To the best of our knowledge, we are the first to use a doodle-style transfer method for synthetic cell image generation. This framework is quite modular and could be applied to other cell image generation problems. A statistical evaluation has shown that features of real and synthetic cell images followed roughly the same distribution. Finally, the realism of the synthetic cell images has been assessed through a visual evaluation performed with the help of medical experts. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

Keywords: bladder cancer; bright-field microscopy; deep learning; style transfer; synthetic cell images; urinary cytology.

Figure 1

Crop of urinary cytology digital slide (a), examples of one healthy urothelial cell (b), and one atypical urothelial cell (c) with their segmentation masks. [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 2

Summary diagram illustrating the process for cell segmentation masks generation. During the learning stage (a), training nucleus and cytoplasm shape descriptors are computed from the training cell segmentation masks and the joint probability density function of these descriptors is approximated with a density $\widehat{f_d}$. During the sampling stage (b), $\widehat{f_d}$ is used to sample new shape descriptors $\tilde{\mathit{d}}$ that are inverted to get new cell segmentations masks. [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 3

Summary diagram of the doodle‐style transfer technique. [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 4

Examples of generated healthy urothelial segmentation masks (a) and generated atypical urothelial segmentation masks (b) by sampling and inverting elliptical Fourier shape descriptors as described in Section 2.2.

Figure 5

Hyperparameters of the doodle‐style transfer technique used for both cell classes.

Figure 6

Examples of synthetic healthy urothelial cell (1st → 3rd line, 1st → 4th column) and synthetic atypical urothelial cells (1st → 3rd line, 5th → 8th column) generated from the presented framework. From left to right: real cell image x_s, real cell segmentation mask m_s, synthetic cell segmentation mask m_t generated with the method described in Section 2.2 and synthetic cell x_t generated with the doodle‐style transfer method described in Section 2.3.2. From line 4th to line 7th, other types of objects are presented such as Malpighian cells (4th line), polynuclear neutrophils (5th line), cells with different staining (HES) / Papanicolaou staining protocol (6th line) and cells scanned with a different illumination technique (FITC fluorescence) (7th line). [Color figure can be viewed at http://wileyonlinelibrary.com]