Self-Attentive Adversarial Stain Normalization

Abstract

Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utilized for biopsy visualization-based diagnostic and prognostic assessment of diseases. Variation in the H&E staining process across different lab sites can lead to significant variations in biopsy image appearance. These variations introduce an undesirable bias when the slides are examined by pathologists or used for training deep learning models. Traditionally proposed stain normalization and color augmentation strategies can handle the human level bias. But deep learning models can easily disentangle the linear transformation used in these approaches, resulting in undesirable bias and lack of generalization. To handle these limitations, we propose a Self-Attentive Adversarial Stain Normalization (SAASN) approach for the normalization of multiple stain appearances to a common domain. This unsupervised generative adversarial approach includes self-attention mechanism for synthesizing images with finer detail while preserving the structural consistency of the biopsy features during translation. SAASN demonstrates consistent and superior performance compared to other popular stain normalization techniques on H&E stained duodenal biopsy image data.

Keywords: Adversarial Learning; Stain Normalization.

Fig. 1.

(Left) H&E stained duodenal biopsy patches created from whole slide images sourced from different locations. (Right) Visual example of a many-to-one stain transfer network. Two different stains are present as inputs within X: X⁽¹⁾ and X⁽²⁾. Both of these domains are translated to Y with G_XY. To complete the cycle, G_YX returns the image back to the X domain, but it can no longer be mapped directly to the input sub-domains X⁽¹⁾ or X⁽²⁾ from which it originated. Instead, the image is mapped back to $\widehat{X}$ which is represents a new domain of stain appearance.

Fig. 2.

Left: Results when mapping was done from two sub-domains of X to Y. Patches from both domains X⁽¹⁾ and X⁽²⁾ are translated to domain Y using G_{Y X}. These generated images are then translated back to a new domain defined by a G_{X Y} as a combination of stain distributions of sub-domains of X. Patches on either end of the second column are real images from domain Y and have been added to visually show the performance of G_{Y X}. Right: Results when mapping was learnt using a single domain in X to Y.

Fig. 3.

Visual comparison of performance in cases where Macenko and Vahadane techniques struggle to properly transfer stain in each scenario. The target image only applies to the Macenko and Vahadane techniques.

Fig. 4.

Visual and quantitative comparison of performance between StainGAN and ablation study on SAASN. The numbers indicate the overall mean ± standard deviation of the SSIM index for the transformation. All models were trained in a many-to-one setup.

Fig. 5.

Normalized Whole Slide Image using ours and traditional approaches. Macenko was chosen because it performed better than Vahadane on our dataset. The target slide for Macenko was empirically selected to give the best translation.