Artifact Augmentation for Enhanced Tissue Detection in Microscope Scanner Systems

Abstract

As the field of routine pathology transitions into the digital realm, there is a surging demand for the full automation of microscope scanners, aiming to expedite the process of digitizing tissue samples, and consequently, enhancing the efficiency of case diagnoses. The key to achieving seamless automatic imaging lies in the precise detection and segmentation of tissue sample regions on the glass slides. State-of-the-art approaches for this task lean heavily on deep learning techniques, particularly U-Net convolutional neural networks. However, since samples can be highly diverse and prepared in various ways, it is almost impossible to be fully prepared for and cover every scenario with training data. We propose a data augmentation step that allows artificially modifying the training data by extending some artifact features of the available data to the rest of the dataset. This procedure can be used to generate images that can be considered synthetic. These artifacts could include felt pen markings, speckles of dirt, residual bubbles in covering glue, or stains. The proposed approach achieved a 1-6% improvement for these samples according to the F₁ Score metric.

Keywords: U-Net; augmentation; classification; convolutional neural network; deep learning; digital microscope scanner; digital pathology; segmentation; tissue detection.

Figure 1

An illustration of a 2-step tissue detection protocol. First, a classification algorithm identifies the sample by selecting the most probable class from a predefined set on which the model was trained. Then, the class-type specific segmentation algorithm detects the exact location of the specimen on the slide by outlining it with a contour mask. In this protocol, each class has a corresponding segmentation algorithm which is the most optimal in detecting the morphological features of the given class.

Figure 2

The figure illustrates that many artifacts may ruin finding the right focus depth in whole slide imaging. The top two blocks show the relative position of the scanning device’s optics and the histology slide with its different layers, i.e., glass slide; specimen; cover glass; covering glue; possible air bubbles in the glue; and dirt, fingerprint, and marker artifacts on the glass surfaces. The bottom block shows the top view of the histology slide. The illustration is not to scale.

Figure 3

The overlay augmentation of imaging artifacts (specks of dirt, felt pen marks, bubbles) may significantly increase the accuracy of tissue detection in histology slides. The drawing illustrates the results of a tissue detection algorithm with and without overlay augmentation of various artifacts on the left-, and on the right-hand side, respectively.

Figure 4

Accurate segmentation and missegmentation of the objects. (a) Accurate segmentation. (b) Felt pen error. (c) Bubble error.

Figure 5

Labeling of the overlay. (a) Original image. (b) Mask of felt pen.

Figure 6

Steps of overlay augmentation. (a) Without augmentation. (b) Scaling. (c) Rotation. (d) Mirroring. (e) Changing of hue. (f) Scratch. (g) Perlin noise. (h) Intensity scaling.

Figure 6

Steps of overlay augmentation. (a) Without augmentation. (b) Scaling. (c) Rotation. (d) Mirroring. (e) Changing of hue. (f) Scratch. (g) Perlin noise. (h) Intensity scaling.

Figure 7

Artifact addition to an originally clean preview image. (a) Part of the input image. (b) Felt pen overlay. (c) Bubble overlay. (d) Part of the output image.

Figure 8

The U-Net architecture consists of a contracting path of 7 layers that captures contextual information and a symmetric expansive path of another 7 layers that recovers spatial details. The layers are interconnected with 2D max-pooling in the contracting branch, and we applied 2D convolutional transpose with batch normalization between the layers on the expanding branch.

Figure 9

Illustration of the detection results, trained with the original training set (first and third column) and extended with the synthetic samples (second and fourth column).

Figure 10

Illustration of cases where detection is still not performed correctly using overlay augmentation: (a) part of the bubble was identified as tissue, (b–d) missed sample—detected as bubble, (e) missed sample—detected as felt-tip pen, (f) missed sample—detected as dirt.