Digital pathology and computational image analysis in nephropathology

Abstract

The emergence of digital pathology - an image-based environment for the acquisition, management and interpretation of pathology information supported by computational techniques for data extraction and analysis - is changing the pathology ecosystem. In particular, by virtue of our new-found ability to generate and curate digital libraries, the field of machine vision can now be effectively applied to histopathological subject matter by individuals who do not have deep expertise in machine vision techniques. Although these novel approaches have already advanced the detection, classification, and prognostication of diseases in the fields of radiology and oncology, renal pathology is just entering the digital era, with the establishment of consortia and digital pathology repositories for the collection, analysis and integration of pathology data with other domains. The development of machine-learning approaches for the extraction of information from image data, allows for tissue interrogation in a way that was not previously possible. The application of these novel tools are placing pathology centre stage in the process of defining new, integrated, biologically and clinically homogeneous disease categories, to identify patients at risk of progression, and shifting current paradigms for the treatment and prevention of kidney diseases.

Fig. 1. The analytical phases of digital pathology.

The pre-analytical phase includes steps involved in tissue procurement, processing and fixation. The analytical phase involves a histology phase (which includes selection of the stain to be used, optimization and validation of the staining procedure) and a digital phase (which includes scanning the slides as whole slide images (WSIs) and curation of the digital library). The post-analytical phase involves data extraction, analysis and interpretation of results. Data extraction is accomplished using human visual analysis and/or machine vision, using human–machine synergistic protocols. Data extracted from the images using human and machine vision are integrated and reported. Data integration can be achieved using computational tools as well as human intuition and domain expertise.

Fig. 2. Machine vision technology as a support tool in nephropathology.

Machine vision tools — including discovery pathomics with deep learning and hand-crafted pathomics — can be used to convert digital pathology images into minable data and provide support to pathologists. a | Machine vision can be used to automatically detect histological primitives. For example, a convolutional neural network can be used to detect glomeruli in frozen kidney sections stained with haematoxylin and eosin. b | Machine vision has also been used to establish a classifier for glomerular disease stage in patients with diabetic glomerulosclerosis. The left panel shows a paraffin-embedded section of a normal glomerulus (top) and a glomerulus with nodular glomerulosclerosis (bottom) stained with periodic acid Schiff. The middle panel shows detection of the cellular/nuclei (blue) and matrix (red) component of the normal (top) and diseased (bottom) glomerulus. The right panel shows the measurements of the glomerular characteristics for both the normal and the diseased glomerulus. c | Application of artificial intelligence-guided morphometry can provide quantitative assessment of the interstitial fractional space. On the left is a paraffin-embedded section stained with trichrome. The right panel shows the automatic detection of interstitial fractional space. The superimposition of a digital grid on the image enables digital morphometry. d | Machine vision can also be used to build models to aid prognostication. For example, qualitative and quantitative automatic detection of features of acute tubular injury may predict the course of the disease and response to therapy: the presence of only a few areas of vacuolization with specific qualitative characteristics could predict rapid recovery from an episode of acute renal failure, with normalization of serum creatinine levels, compared with a renal biopsy containing much greater levels of vacuolization. e | Computational imaging tools can be combined with other methodologies for parallel discovery. For example, computational image analysis tools can be applied to guide laser capture microdissection by identifying structures with similar pathomic signatures within the same biopsy sample and across biopsy samples. The structures with similar pathomic signatures can be captured and analysed separately, allowing for spatial mapping of pathogenomic signatures. Panel b reprinted courtesy of P. Sander and B. Ginley, University at Buffalo, NY, USA. Panel c reprinted courtesy of J. Hodgin, University of Michigan, MI, USA.

Fig. 3. The nephropathology digital ecosystem.

The digital ecosystem covers three phases: the digital pathology phase (analogue to digital conversion), the knowledge extraction phase, which relies on human and/or artificial intelligence (AI), and the actionable intelligence phase, in which integrated knowledge is applied to patient care. Each phase begins with input data and ends by generating output data that represent the input for the successive phase. In the digital pathology phase, glass slides from the renal biopsy sample (that is, the analogue input data), are converted into whole slide images (WSIs) (that is, digital output data). The WSIs represent the input data for the knowledge extraction phase, from which useful information is generated (output data). Knowledge extraction can be broken down into two types: human cognition and AI. AI techniques can be implemented as companion diagnostic tools alongside human cognition. Human cognition is employed when WSIs (input image data) are visually assessed or scored by a trained pathologist to generate diagnoses or morphological profiles as digital pathology-derived knowledge (output data). AI-based machine vision comprises both hand-crafted pathomics and discovery pathomics. For hand-crafted pathomics, image data (input) is transcribed into pathomic signatures (the pathomic feature space) and then translated into digital pathology-derived knowledge (the output data) using machine-learning models. For discovery pathomics, the image data (the input data) are transcribed or encoded into structured data and then decoded into an output signal (the output image data) using deep learning. Finally, in the actionable intelligence phase, the knowledge obtained from the digital images (input data) is integrated with other data types, for example, omics and clinical data, and used to diagnose, prognosticate and select targeted treatments (output data) for the patient.