Digital Histopathology by Infrared Spectroscopic Imaging

Abstract

Infrared (IR) spectroscopic imaging records spatially resolved molecular vibrational spectra, enabling a comprehensive measurement of the chemical makeup and heterogeneity of biological tissues. Combining this novel contrast mechanism in microscopy with the use of artificial intelligence can transform the practice of histopathology, which currently relies largely on human examination of morphologic patterns within stained tissue. First, this review summarizes IR imaging instrumentation especially suited to histopathology, analyses of its performance, and major trends. Second, an overview of data processing methods and application of machine learning is given, with an emphasis on the emerging use of deep learning. Third, a discussion on workflows in pathology is provided, with four categories proposed based on the complexity of methods and the analytical performance needed. Last, a set of guidelines, termed experimental and analytical specifications for spectroscopic imaging in histopathology, are proposed to help standardize the diversity of approaches in this emerging area.

Keywords: chemical imaging; deep learning; digital pathology; imaging; infrared spectroscopy; machine learning.

Figure 1

Current histopathologic evaluation process and the proposed process based on IR imaging. Tissue is first obtained, fixed, and embedded in a cutting medium, and a thin section is obtained on a slide as the sample. (Step 1) The sample is stained, commonly with H&E. (Step 2) Optical microscopy is the mainstay for visualizing tissue morphology to diagnose disease or for research assessments, providing (Step 3) images in which contrast is enhanced by the stain. (Step 4a) Images are assessed by a pathologist. (Step 4b) Digital pathology, wherein computer algorithms are used to assist the pathologist by quantitative morphological information, is an emerging aid. (Step 5) Tissue evaluations can then provide input for clinical diagnoses or research insights. In contrast, steps 6 and 7 show the workflow possible with IR imaging. (Step 6) An IR microscope, providing the benefits of microscopy and spectroscopy, integrally uses a computer because of the large volume of data and impracticality of manual interpretation. The data have two spatial dimensions, similar to optical microscopy, but a much larger spectral dimension. Machine learning is applied in two important areas: (Step 7a) stainless staining, to reproduce the stained images commonly used in pathology, and (Steps 7b and 7c) chemical histopathology, to gain novel information over present methods and visualization that eliminates the need for painstaking examination of tissues. (Step 7b) Segmentation of tissue into epithelial cells and other components (collectively, the stroma), and (Step 7c) comprehensive examination of breast tissue for disease and stromal reaction. In Step 6, the image is adapted from Reference and the spectrum is adapted from Reference . Abbreviations: DCIS, ductal carcinoma in situ; Desmo., desmoplastic stroma; H&E, hematoxylin and eosin; IBC, invasive breast cancer; IR, infrared.

Figure 2

IR spectroscopic imaging measurements and use. (a) Theoretical prediction of the smallest pixel size to achieve the highest spatial fidelity. Panel a adapted from Reference . The graph provides a guide to designing IR imaging systems. (b) A custom-built DFIR imaging system, showing the essential components of an IR imaging system. (c) Evaluation of the spatial quality of the imaging system with two different objective lenses. (d) Spatial (top) and spectral (bottom) performance can be quantified. Panels b–d adapted from Reference . (e) Augmented performance of optimal optical design (i) compared with that of commercial systems (ii). Use of a solid immersion lens can increase the quality of images, providing higher resolution (iii) than standard images (iv). Using a hybrid microscopy format can provide optical microscopy resolution (v) compared with the optimal all-IR resolution (vi). Panel e, subpanels v and vi, adapted from Reference . ( f ) FT-IR imaging data provide high-quality spectral and spatial information that can provide color-coded pictures of the colon tumor microenvironment. (g) Statistical accuracy of detecting tumor and microenvironmental cells. (h) Prediction of risk for moderate-grade tumors. Panels f–h adapted from Reference . Abbreviations: Abs., absorbance; AUC, area under the curve; DAQ, data acquisition; DFIR, discrete frequency IR; FT-IR, Fourier transform–IR; H&E, hematoxylin and eosin; HR, hazard ratio; IR, infrared; MCT, mercury-cadmium-telluride; NA, numerical aperture; Preamp.; pre-amplifier; QCL, quantum cascade laser.

Figure 3

Study design and data workflows in infrared (IR)-based pathology. (a) General idea of a study design. A population of specific disease states is identified from several clinical systems for the study to assure a diversity of patients and practices. Representative cases for IR imaging are identified and tissue samples are prepared, often building in sampling redundancy (i.e., sampling from the same patients), use of matched cases (from the same patient or matching for known variables), and high statistical numbers. From each sample used in the study, an IR imaging data set is obtained. (b) A computational pipeline is then devised to assess the use of a histopathology model along with analytical parameters to process the data and extract information, with statistical validation. Multistep processing workflows are designed for specific cases as needed and benefit from the opportunity to include substantial spectroscopy and pathology knowledge. Colors of the population indicate the natural variation as well as variation due to disease, while variation introduced by clinical settings and sampling is indicated by additional colors.

Figure 4

Designer workflows and use of deep learning for histopathology. Carefully crafted studies focus on a specific problem with custom design and statistical measures that relate back to the imaging data. (a) A well-crafted workflow for classification clearly listing the study design and experimental steps, including statistically valid results with independent validation. One workflow distinguishes healthy from cancer samples (purple boxes). The middle workflow (blue boxes) distinguishes MSS patients from MSI-H ones. The IR analyses are validated by clinical molecular analyses (gray boxes, right). (b) Explicit differences between MSS and MSI-H cohorts for training/testing (40 samples, 21 MSS and 19 MSI-H) and for verification (60 samples). The gray dashed line represents a threshold (63%) that segments the two groups. (c) Receiver operating curve with AUC demonstrating high accuracy. (d) Projection of classification back to images (top; IR images in which MSS is indicated by blue and MSI-H by orange) and comparison with H&E images (bottom), demonstrating the ease of conveying information with IR-classified images. Panels a–d adapted from Reference , with permission from the authors. (e) Deep learning can be used to (i) segment images in one step, (ii) increase speed by estimating missing data, and (iii) use multimodal information to super-resolve images using data-driven algorithms. Panel e adapted from Reference ; copyright 2021 Elsevier. Abbreviations: Adv, adversarial; AUC, area under the curve; DFIR, discrete frequency IR; H&E, hematoxylin and eosin; IHC, immunohistochemistry; IR, infrared; IR-REC, IR-reconstruction; IR-SEG, IR-segmentation; IR-SR, IR super-resolution; MSE, mean square error; MSI-H, high microsatellite instable; MSS, microsatellite stable; PCR, polymerase chain reaction; RF, random forest; ROI, region of interest; VGG, visual geometry group.