Deep learning in cancer pathology: a new generation of clinical biomarkers

Abstract

Clinical workflows in oncology rely on predictive and prognostic molecular biomarkers. However, the growing number of these complex biomarkers tends to increase the cost and time for decision-making in routine daily oncology practice; furthermore, biomarkers often require tumour tissue on top of routine diagnostic material. Nevertheless, routinely available tumour tissue contains an abundance of clinically relevant information that is currently not fully exploited. Advances in deep learning (DL), an artificial intelligence (AI) technology, have enabled the extraction of previously hidden information directly from routine histology images of cancer, providing potentially clinically useful information. Here, we outline emerging concepts of how DL can extract biomarkers directly from histology images and summarise studies of basic and advanced image analysis for cancer histology. Basic image analysis tasks include detection, grading and subtyping of tumour tissue in histology images; they are aimed at automating pathology workflows and consequently do not immediately translate into clinical decisions. Exceeding such basic approaches, DL has also been used for advanced image analysis tasks, which have the potential of directly affecting clinical decision-making processes. These advanced approaches include inference of molecular features, prediction of survival and end-to-end prediction of therapy response. Predictions made by such DL systems could simplify and enrich clinical decision-making, but require rigorous external validation in clinical settings.

Fig. 1. Consensus pipeline of deep learning in pathology.

a Routine histology image of lung cancer (from The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA)). b Size comparison (in terms of pixels) of a chest CT scan of the same patient. c Consensus image-processing pipeline. First, either the whole slide or just the tumour region is tessellated into smaller image tiles. d These tiles comprise an image library, similar to the library preparation (prep.) in genome sequencing. e Tiles are preprocessed to achieve rotational constancy and augment the dataset. f Deep- learning classifiers are developed and deployed by splitting the patient cohort into a training and testing set, by using cross-validation or by having multiple cohorts available for training and testing. g Ideally, an additional external dataset is used for validation of the resulting classifier.

Fig. 2. Clinical applications of basic and advanced deep-learning (DL) image analysis in histopathology.

DL pathology can be applied to tumour detection and identification of subtype (basic applications) or to predict clinical features of interest (advanced application). Published studies (indicated by reference number) are classified according to the level of evidence (monocentric (internally approved), multicentric (externally approved) or FDA approved). a Basic image analysis tasks, including tumour detection, grading and subtyping. b Advanced image analysis tasks, including those that exceed pathologists’ routine capacities, such as prediction of mutation, prognosis and response. AI   artificial intelligence, NSCLC    non-small-cell lung cancer, WSI   whole-slide image, ER   oestrogen receptor, MSI   microsatellite instability, GI   gastrointestinal, SPOP   speckle-type BTB/POZ protein, BAP1   BRCA-associated protein 1, HNSCC   head and neck squamous cell carcinoma, CCA   cholangiocarcinoma.