MIHIC: a multiplex IHC histopathological image classification dataset for lung cancer immune microenvironment quantification

Abstract

Background: Immunohistochemistry (IHC) is a widely used laboratory technique for cancer diagnosis, which selectively binds specific antibodies to target proteins in tissue samples and then makes the bound proteins visible through chemical staining. Deep learning approaches have the potential to be employed in quantifying tumor immune micro-environment (TIME) in digitized IHC histological slides. However, it lacks of publicly available IHC datasets explicitly collected for the in-depth TIME analysis.

Method: In this paper, a notable Multiplex IHC Histopathological Image Classification (MIHIC) dataset is created based on manual annotations by pathologists, which is publicly available for exploring deep learning models to quantify variables associated with the TIME in lung cancer. The MIHIC dataset comprises of totally 309,698 multiplex IHC stained histological image patches, encompassing seven distinct tissue types: Alveoli, Immune cells, Necrosis, Stroma, Tumor, Other and Background. By using the MIHIC dataset, we conduct a series of experiments that utilize both convolutional neural networks (CNNs) and transformer models to benchmark IHC stained histological image classifications. We finally quantify lung cancer immune microenvironment variables by using the top-performing model on tissue microarray (TMA) cores, which are subsequently used to predict patients' survival outcomes.

Result: Experiments show that transformer models tend to provide slightly better performances than CNN models in histological image classifications, although both types of models provide the highest accuracy of 0.811 on the testing dataset in MIHIC. The automatically quantified TIME variables, which reflect proportions of immune cells over stroma and tumor over tissue core, show prognostic value for overall survival of lung cancer patients.

Conclusion: To the best of our knowledge, MIHIC is the first publicly available lung cancer IHC histopathological dataset that includes images with 12 different IHC stains, meticulously annotated by multiple pathologists across 7 distinct categories. This dataset holds significant potential for researchers to explore novel techniques for quantifying the TIME and advancing our understanding of the interactions between the immune system and tumors.

Keywords: database; image classification; immunohistochemical image; lung cancer; transformer models.

Figure 1

Overview of our study.

Figure 2

Illustration of MIHIC dataset creation. (A) Manually annotated tissue cores, (B) different regions of interest (ROIs), (C) image patch extraction.

Figure 3

Examples of 7 histological image types included in the MIHIC dataset.

Figure 4

The comprehensive pipeline of quantitative analysis for TMA sections. (A) Tissue core extraction, (B) extracted tissue core samples, (C) diagram of TIME quantification and survival analysis.

Figure 5

Tissue identification in a tissue core. (A) CD3 tissue core, (B) tissue classification result.

Figure 6

Confusion matrix of different CNN models.

Figure 7

Confusion matrix of different Transformer models.

Figure 8

ROC curves of multi-class classification using various deep learning models. (A) ROC curves for different CNN models, (B) ROC curves for different Transformer models.

Figure 9

Box plots of quantified TIME variables. (A) ln(Immune cells/Tumor), (B) ln(Immune cells/Stroma), (C) ln(Immune cells/Necrosis), (D) ln(Tumor/Stroma), (E) ln(Stroma/Tissue core), (F) ln(Immune/Tissue core), (G) ln(Tumor/Tissue core).

Figure 10

Survival analysis visualization. (A) Immune cells/Stroma distribution density and cutoff point selection, (B) Kaplan-Meier survival curves based on Immune cells/Stroma, (C) Cumulative hazard based on Immune cells/Stroma, (D) Tumor/Tissue core distribution density and cutoff point selection, (E) Kaplan-Meier survival curves based on Tumor/Tissue core, (F) Cumulative hazard based on Tumor/Tissue core.