Artificial intelligence and machine learning for medical imaging: A technology review

Abstract

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.

Keywords: Artificial intelligence; Deep learning; Machine learning; Medical imaging.

Figure 1.

Artificial intelligence, machine learning, and deep learning can be seen as matryoshkas nested in each other. Artificial intelligence gathers both symbolic (top down) and connectionist (bottom up) approaches. Machine learning is the dominant branch of connectionism, combining biological (neural networks) and statistical (data-driven learning theory) influences. Deep learning focuses mainly on large-size neural networks, with functional specificities to process images, sounds, videos, etc.

Figure 2.

Three classical learning frameworks in artificial intelligence: supervised, semi-supervised, and unsupervised learning. Supervised learning relies on known input-output pairs. If some output labels are difficult or expensive to get, semi-supervised learning can apply. If no labels are available, unsupervised learning allows for a more exploratory approach of data.

Figure 3.

The tight framework of supervised learning can be hybridized with unsupervised learning to make room for practical cases and problems, as well as to accommodate temporality. Delaying supervision in future times leads towards reinforcement learning. Incompletely labelled data fosters semi-supervised learning, whereas small data sets encourage reusing (parts of) models trained previously on similar but bigger data sets, like in transfer learning. In self-supervision, pretraining relies on solving dummy supervised problems, where fake labels are created based on the inherent structure of image or sound data.

Figure 4.

General ML pipeline for supervised learning: supervised predictive models are fed with features that are extracted and/or selected beforehand in an unsupervised way. Feature selection can, however, be embedded in some models, using regularization, for instance; selection then becomes supervised and therefore often improved. Classical (shallow) models tend to critically depend on unsupervised feature extraction and selection to preprocess data. In contrast, deep learning drops unsupervised feature extraction and selection; instead, it embeds multiple trainable layers of feature extractors and selectors, allowing the full pipeline to be supervised, end to end.

Figure 5.

Artificial neural networks in a nutshell. (a) The formal neuron, processing several dendritic inputs through a nonlinear activation function f, to produce its actional output. (b) The neurons can be interconnected in a feed-forward way, into successive layers; as soon as a nonlinear ‘hidden’ layer is inserted in between the inputs and outputs, the network can potentially approximate any function; specific activation functions can be fitted in the output layer to achieve either regression or classification. (c) Examples of nonlinear activation functions in the hidden layers: the step function, from biological inspiration, the sigmoid, its continuous and differentiable surrogate, and the rectified linear unit (ReLU), that improves training of deep layers.

Figure 6.

(a) Decision trees assign labels (leafs) to a given sample by going through a multi-level structure where different features (root nodes) and solutions (branches) are tested. (b) In a Random Forest algorithm, decision trees are combined, following an ensemble learning approach, which enables to get more accurate predictions than a single tree. Each individual tree in the forest spits out a class prediction and the class with the most votes becomes the final model’s prediction.

Figure 7.

Principle of the linear support vector machine, which lifts the indeterminacy of separable classification by fitting the thickest margin, stuck in between a few ‘support vectors’. The principle can be extended to nonlinear class separation by using Mercer kernels [125].

Figure 8.

Number of publications since 2010 till 2020 in the PubMed repository, containing keywords related to AI/ML/DL methods in the title and/or abstract.

Figure 9.

Typical architecture for a (deep) Convolutional Neural Network (CNN). Different convolutional kernels scan the input images leading to several feature maps. Then, down-sampling operations, such as max-pooling (i.e., taking the maximum value of a block of pixels), are applied to reduce the size of the feature maps. These two operations, convolution and pooling, are applied multiple times to extract higher-level features. At the end, the feature maps are flattened and passed through fully connected layers of neurons (see Figure 5), to obtain a final prediction. The embedded (automatic and unsupervised) feature extraction (Figure 4) is what enables CNNs to remove all handcrafted operations and makes them so powerful.

Figure 10.

Structure of Generative Adversarial Networks (GANs). Starting from random noise, the generator (G) uses the feedback from the discriminator (D) and learns to create images that are similar to the provided ground truth.