Artificial intelligence and machine learning in nephropathology

Abstract

Artificial intelligence (AI) for the purpose of this review is an umbrella term for technologies emulating a nephropathologist's ability to extract information on diagnosis, prognosis, and therapy responsiveness from native or transplant kidney biopsies. Although AI can be used to analyze a wide variety of biopsy-related data, this review focuses on whole slide images traditionally used in nephropathology. AI applications in nephropathology have recently become available through several advancing technologies, including (i) widespread introduction of glass slide scanners, (ii) data servers in pathology departments worldwide, and (iii) through greatly improved computer hardware to enable AI training. In this review, we explain how AI can enhance the reproducibility of nephropathology results for certain parameters in the context of precision medicine using advanced architectures, such as convolutional neural networks, that are currently the state of the art in machine learning software for this task. Because AI applications in nephropathology are still in their infancy, we show the power and potential of AI applications mostly in the example of oncopathology. Moreover, we discuss the technological obstacles as well as the current stakeholder and regulatory concerns about developing AI applications in nephropathology from the perspective of nephropathologists and the wider nephrology community. We expect the gradual introduction of these technologies into routine diagnostics and research for selective tasks, suggesting that this technology will enhance the performance of nephropathologists rather than making them redundant.

Keywords: artificial intelligence; computer; convolutional neural network; image recognition; nephropathology.

Figure 1 |. Nephropathology in the context of other disciplines contributing to big data in precision medicine.

Artificial intelligence can be used in all steps of tissue analysis and during the integration and analysis of data from other disciplines for the big data–based determination of diagnosis, prognosis, and treatment of an individual patient. IF, immunofluorescence; LM, light microscopy; Multi-IF, multichannel immunofluorescence; TEM, transmission electron microscopy.

Figure 2 |. Development of machine learning algorithms and their architecture and functionality.

(a) The first neural networks used in image analysis were the simple perceptrons with a single input layer connected to a binary output. (b) Radial basis forward networks represent an intermediate stage with a hidden layer in between the input and output. (c) The current state of the art in the analysis of whole slide images is the convolutional neural network (CNN) that links the pixel input to the classification output via several interconnected hidden layers as shown in this fictional example for the classification of severity of T cell–mediated rejection (TCMR), antibody-mediated rejection (AMR), and scar in a rodent kidney allograft. Note that the hidden layers in such CNNs currently remain a black box. However, the trust-building mandatory transparency can be achieved with additional software, as shown in Figure 3. To optimize viewing of this image, please see the online version of this article at www.kidney-international.org.

Figure 3 |. Example of a preliminary convolutional neural network (CNN) solution for a classification task in transplant nephropathology.

The CNN (schematically depicted in the top right) was trained on 279 images from glomerular transections on periodic acid–Schiff (PAS)–stained slides from 6 transplant biopsies with antibody-mediated rejection (AMR) and 6 biopsies without AMR (no AMR). Data augmentation on this relatively small sample size was achieved with rotation. With this semi-supervised training on this augmented data set, the classification accuracy for individual glomeruli reached 91.3%. Interestingly, data augmentation through cropping resulted in lower accuracy. Grad-CAM was used for the superimposition of a heat map highlighting the decisive areas for classification. A trained human nephropathologist would look for the features of glomerulitis, glomerular basement membrane splitting, and perhaps also secondary focal and segmental glomerulosclerosis for this classification task. The misclassification example shows the advantage of the superimposed heat maps. Obviously, the decisive areas (superimposed as a polygon on the PAS image) are largely outside the glomerular tuft, alerting the pathologist of a possible misclassification. In contrast, the heat map in the example with the correct classification highlights tuft segments with glomerulitis (arrows) indicative of AMR. To optimize viewing of this image, please see the online version of this article at www.kidneyinternational.org.