H&E to IHC virtual staining methods in breast cancer: an overview and benchmarking

Abstract

Immunohistochemistry (IHC) is crucial for the clinical categorisation of breast cancer cases. Deep generative models may offer a cost-effective alternative by virtually generating IHC images from hematoxylin and eosin samples. This review explores the state-of-the-art in virtual staining for breast cancer biomarkers (HER2, PgR, ER and Ki-67) and benchmarks several models on public datasets. It serves as a resource for researchers and clinicians interested in applying or developing virtual staining techniques.

Fig. 1. Interpretation of commonly used immunohistochemistry (IHC) markers in breast cancer.

A, B Estrogen receptor (ER); C, D Progesterone receptor (PgR); E, F Ki67; G–J Human epidermal growth factor receptor 2 (HER2). For HER2, an IHC 2+ score requires confirmation by in situ hybridisation (ISH), and an IHC 1+ or 2+ may be reported as “HER2 Low” and are eligible for clinically appropriate HER2-targeted therapy, according to ASCO/CAP 2023 Guideline update. Images adapted from the MIST test set.

Fig. 2. Generative adversarial networks (GANs) basic structures.

a General GAN and (b) conditional GAN, where z is a noise vector, x is a real image from the source distribution, y is a real image from the target distribution, $\widehat{y}$ is the generated image, D is the discriminator, and G is the generator. Example tiles adapted from the BCI dataset.

Fig. 3. CycleGAN framework.

x and y are real images from the source and target distributions, respectively, $\widehat{x}$ and $\widehat{y}$ are generated images, G_XY is the generator from the source to the target domain, G_YX the generator from the target to the source domain, D_X the discriminator for the source domain and D_Y the discriminator for the target domain. The cycle-consistency concept ensures that a generated image can be translated back to the source domain via the opposite generator. Example tiles adapted from the BCI dataset.

Fig. 4. Contrastive unpaired translation (CUT) framework.

x and y are real images from the source and target distributions, respectively, $\widehat{y}$ is the generated image, G_enc is the generator encoder and D is the target discriminator. Example tiles adapted from the BCI dataset.

Fig. 5. Diffusion models’ basic concept.

The diffusion process iteratively adds noise (z), while the denoising process aims to reconstruct the original image (x). Example tile adapted from the BCI dataset.

Fig. 6

PRISMA flowchart illustrating the selection process for the studies included in the review, identified via databases and other sources.

Fig. 7. Benchmarking results examples from the MIST dataset, for ER, PgR, Ki67 and HER2 stainings.

Images with higher resolution are provided in Supplementary Figs. 1–4.

Fig. 8. Benchmarking generalisation results on HER2 examples from the BCI dataset.

Images with higher resolution are provided in the supplementary Fig. 5.