Precision immunoprofiling by image analysis and artificial intelligence

Abstract

Clinical success of immunotherapy is driving the need for new prognostic and predictive assays to inform patient selection and stratification. This requirement can be met by a combination of computational pathology and artificial intelligence. Here, we critically assess computational approaches supporting the development of a standardized methodology in the assessment of immune-oncology biomarkers, such as PD-L1 and immune cell infiltrates. We examine immunoprofiling through spatial analysis of tumor-immune cell interactions and multiplexing technologies as a predictor of patient response to cancer treatment. Further, we discuss how integrated bioinformatics can enable the amalgamation of complex morphological phenotypes with the multiomics datasets that drive precision medicine. We provide an outline to machine learning (ML) and artificial intelligence tools and illustrate fields of application in immune-oncology, such as pattern-recognition in large and complex datasets and deep learning approaches for survival analysis. Synergies of surgical pathology and computational analyses are expected to improve patient stratification in immuno-oncology. We propose that future clinical demands will be best met by (1) dedicated research at the interface of pathology and bioinformatics, supported by professional societies, and (2) the integration of data sciences and digital image analysis in the professional education of pathologists.

Keywords: Artificial intelligence; Digital pathology; Image analysis; Immuno-oncology; Immunotherapy; Machine learning; Personalized medicine.

Fig. 1

Assessment of PD-L1 expression by computational image analysis. a Malignant melanoma of the superficial spreading type stained for PD-L1 by IHC. Digital annotation of the tumor tissue is shown in yellow. Substantial marker heterogeneity and expression of PD-L1 in tumor-infiltrating inflammatory cells complicates conventional histopathological assessment of PD-L1 positivity. b Digital scoring of PD-L1 implemented on the HALO™ platform (Indica labs, Corrales, NM, USA). A total of 34.882 cells were detected by nuclear segmentation using the hematoxylin counterstain for nuclear seeding followed by cell/nuclear boundary detection and postprocessing according to pathologist-controlled cellular parameters, such as nuclear size, roundness, and optical density. Membranous reactivity for 3,3′-diaminobenzidine (DAB) is detected and analyzed according to pathologist-set positivity thresholds. PD-L1 negative stromal cells and normal squamous epithelium serve as on-slide negative controls. In this case, 11.802 PD-L1 positive cells were detected for a total of 33.8% positive cells within the annotation region. c–d PD-L1 reactivity in infiltrating immune cells can skew the assessment of PD-L1 expression in solid tumors with intrinsically low expression levels of PD-L1. e–f Machine learning algorithms trained on large sample sets to differentiate PD-L1 positive immune cells (green) from tumor cell populations (red) represent a powerful approach for tissue classification. Tissue classification is followed by cell-level analysis of DAB expression for the precise assessment of PD-L1 expression in tumor cells only

Fig. 2

Spatial analysis of tumor immune cell infiltration. a Colorectal adenocarcinoma tissue microarray (TMA) spot stained for cytokeratin (Fast Red) and CD8+ T cells (DAB) with hematoxylin as a nuclear counterstain (left). Computational color deconvolution is performed for separate detection of cell nuclei, Fast Red, and DAB reaction products (middle), followed by nuclear segmentation and scoring of all cell populations (tumor cells: red; CD-8+ T cells: brown; marker negative cell nuclei: blue). A total of 1.623 cells were detected in this sample including 867 tumor cells, 330 CD8+ T cells, and 644 marker-negative cells. b Spatial plotting implemented on the HALO™ platform showing the localization of 867 cytokeratin positive tumor cells and 330 CD8+ T cells in this TMA spot. This allows to extract precise data on the relative distribution of T cells to the intraepithelial and stromal compartment in the tumor microenvironment. In this sample, 112 CD8+ T cells (or 33.9%) are localized to the intraepithelial compartment, while 218 CD8+ T cells (or 66.1%) localize to the tumor stroma. c Recording of the x-y coordinates in the tissue sample allows to define cell-cell relations by spatial analysis, such as the definition of nearest neighbor relationships between the tumor and CD8+ cell population (left) as well as the extraction of precise cell-cell distance measures (right)