Machine learning: applications of artificial intelligence to imaging and diagnosis

Abstract

Machine learning (ML) is a form of artificial intelligence which is placed to transform the twenty-first century. Rapid, recent progress in its underlying architecture and algorithms and growth in the size of datasets have led to increasing computer competence across a range of fields. These include driving a vehicle, language translation, chatbots and beyond human performance at complex board games such as Go. Here, we review the fundamentals and algorithms behind machine learning and highlight specific approaches to learning and optimisation. We then summarise the applications of ML to medicine. In particular, we showcase recent diagnostic performances, and caveats, in the fields of dermatology, radiology, pathology and general microscopy.

Keywords: Artificial intelligence; Computer vision; Dermatology; Imaging; Machine learning; Microscopy; Radiology.

Fig. 1

Classification. An example of the individual inputs and probabilistic outputs of a classifier model. The system comprises of a ternary classifier where an image can be either a cat, a dog or a goat. In this example, the system is trained on a set of images which are labelled data as cat (y₃ = [1, 0, 0]), dog (y₁ = [0, 1, 0]) or goat (y₂ = [0, 0, 1]). The classifier then runs on a new test set of data,where it correctly identifies the dog and the cat but erroneously classifies the goat image as a dog

Fig. 2

Stochastic gradient descent. An illustration of stochastic gradient descent for an abstract function R(θ₁, θ₂). Three initial estimates (path 1, blue; path 2, green; path 3, red) are shown that lead to discovery of the same global minimum (paths 1–3). However, one initial estimate and its subsequent path (path 4, yellow) lead to an erroneous local minimum

Fig. 3

Underfitting, overfitting and the bias-variance trade-off. An illustration demonstrating a classification problem (segmenting two data sets, blue and orange). a The raw data set. b An example of underfitting, where a too simple separation has resulted in misclassifying some members of each data set. c An example of a well-fit classifier which correctly separates the data sets and classifies correctly nearly all members of both data sets, without too complex a model. d An overfit classifier, which correctly identifies all the members of each data sets but is overly complex, has high variance, and incorrectly separates the space

Fig. 4

Applying machine learning to medical imaging. a Demonstration of classification and clustering in the final hidden layer of the convolutional neural network in biopsy-proven photographic test sets. Coloured point clouds represent different disease categories, insets show images corresponding to various points. Effective clustering of similar diagnosis can be observed by eye and measured statistically with ROC curves (taken from Esteva et al. 2017). b Computer-aided diagnosis of lymph node metastases in women with breast cancer. The colour scale bar (top right) indicates the probability for each pixel to be part of a metastatic region. The top and bottom rows show two annotated micrometastatic regions in whole-slide images of haematoxylin and eosin-stained lymph node tissue sections taken from the test set of Cancer Metastases in Lymph Nodes Challenge 2016 (CAMELYON 16) dataset. Second through fourth columns show probability maps from each team overlaid on the original image: HMS, Harvard Medical School; MIT, Massachusetts Institute of Technology; MGH, Massachusetts General Hospital and CULab, Chinese University Lab. The top two deep learning-based systems, from the teams HMS and MITII and HMS and MGH III outperformed all the pathologists without time constraint in this study (taken from Beijnordi et al. 2017). c Trainable WEKA segmentation pipeline for pixel classification. Image features are first extracted using native non-machine learning methods inside the imaging software Fiji [ref] (i). One example of such a method is edge detection (ii) through the Canny edge detection which is based on analysis of gradients. Next, a WEKA (Hall et al. 2009) learning scheme is trained on a set of pixel samples represented as feature vectors (from various image features), and the user provides iterative and interactive feedback to correct or add labels. This is then used for semantic segmentation of the image (iii) and finally object identification (iv). An example pipeline showing a serial section from transmission electron microscopy of Drosophila larva ventral nerve cord with pixels divided into three classes: membrane, mitochondria and cytoplasm (bottom) (taken from Arganda-Carreras et al. 2017)