Autofluorescence Virtual Staining System for H&E Histology and Multiplex Immunofluorescence Applied to Immuno-Oncology Biomarkers in Lung Cancer

Abstract

Abstract: Virtual staining for digital pathology has great potential to enable spatial biology research, improve efficiency and reliability in the clinical workflow, as well as conserve tissue samples in a nondestructive manner. In this study, we demonstrate the feasibility of generating virtual stains for hematoxylin and eosin (H&E) and a multiplex immunofluorescence (mIF) immuno-oncology panel (DAPI, PanCK, PD-L1, CD3, and CD8) from autofluorescence (AF) images of unstained non–small cell lung cancer tissue by combining high-throughput hyperspectral fluorescence microscopy and machine learning. Using domain-specific computational methods, we evaluated the accuracy of virtual H&E staining for histologic subtyping and virtual mIF for cell segmentation–based measurements, including clinically relevant measurements such as tumor area, T-cell density, and PD-L1 expression (tumor proportion score and combined positive score). The virtual stains reproduce key morphologic features and protein biomarker expressions at both tissue and cell levels compared with real stains, enable the identification of key immune phenotypes important for immuno-oncology, and show moderate to good performance across various evaluation metrics. This study extends our previous work on virtual staining from AF in liver disease and prostate cancer, further demonstrating the generalizability of this deep learning technique to a different disease (lung cancer) and stain modality (mIF).

Significance: We extend the capabilities of virtual staining from AF to a different disease and stain modality. Our work includes newly developed virtual stains for H&E and a multiplex immunofluorescence panel (DAPI, PanCK, PD-L1, CD3, and CD8) for non-small cell lung cancer, which reproduce the key features of real stains.

Figure 1

Comparison of current pathology workflow and virtual staining workflow. Multiple stains are achieved by chemical staining of multiple sections in the current workflow and by concurrent virtual staining of a single section in the proposed workflow.

Figure 2

pIHC and segmentation algorithms were developed for mIF model development and evaluation. A, Examples of mIF (top) and the corresponding pIHC (bottom) for PanCK, PD-L1, CD3, and CD8. As per conventional practice, the residual unmixed AF is not shown in the mIF to maintain appropriate visual quality. B, Example of the segmentations of tumor regions based on the PanCK stain (dashed outline, green) and negative (solid outline, blue) and positive (solid outline, respective color) cells for PD-L1, CD3, and CD8.

Figure 3

A, Example of real (left) and virtual (right) stains for a H&E WSI. B, Magnification series of real (left) and virtual (right) stains for concentric regions from the WSI at 10× (top), 20× (middle), and 40× (bottom) magnifications. C, Example of the segmentations on real (left) and virtual (right) stains obtained from automatic histologic subtyping.

Figure 4

A, Example of real (left) and virtual (right) stains showing the WSI composite for all mIF stains. B–E, Examples of real (left) and virtual (right) stains from the individual models for DAPI + PanCK, PD-L1, CD3, and CD8 at several regions across the WSI.

Figure 5

Examples of real (left) and virtual (right) stains showing key immune phenotypes. A, PD-L1–positive tumor. B, PD-L1–negative tumor. C, Immune-inflamed tumor with T-cell infiltration. D, Immune-excluded tumor with T-cell accumulation in surrounding stroma. E, Immune-desert tumor with no T cells.