Radiomics with artificial intelligence: a practical guide for beginners

Abstract

Radiomics is a relatively new word for the field of radiology, meaning the extraction of a high number of quantitative features from medical images. Artificial intelligence (AI) is broadly a set of advanced computational algorithms that basically learn the patterns in the data provided to make predictions on unseen data sets. Radiomics can be coupled with AI because of its better capability of handling a massive amount of data compared with the traditional statistical methods. Together, the primary purpose of these fields is to extract and analyze as much and meaningful hidden quantitative data as possible to be used in decision support. Nowadays, both radiomics and AI have been getting attention for their remarkable success in various radiological tasks, which has been met with anxiety by most of the radiologists due to the fear of replacement by intelligent machines. Considering ever-developing advances in computational power and availability of large data sets, the marriage of humans and machines in future clinical practice seems inevitable. Therefore, regardless of their feelings, the radiologists should be familiar with these concepts. Our goal in this paper was three-fold: first, to familiarize radiologists with the radiomics and AI; second, to encourage the radiologists to get involved in these ever-developing fields; and, third, to provide a set of recommendations for good practice in design and assessment of future works.

Figure 1

Venn diagram of the concepts related to artificial intelligence (AI). AI is the simulation of human intelligence processes like learning, reasoning, and self-correction by the machines, particularly the computer systems. AI is a broad concept that covers many machine learning techniques such as k-nearest neighbors, support vector machine, decision trees, and neural networks. Neural networks include various algorithms ranging from very simple to complex architectures, such as multi-layer perceptron and deep learning or convolutional neural networks.

Figure 2

Over-simplified representation of traditional and deep learning-based radiomics. Representative CT and MRI images in Fig. 2 and Fig. 3 are obtained from the Cancer Imaging Archive (TCIA), specifically from the collections of TCGA-KIRC (72, 73) and LGG-1p19qDeletion (–75), which are publicly and freely available.

Figure 3 .a–c

Different image types for radiomic feature extraction: (a), original image; (b), filtered image; (c), wavelet-transformed images. Representative CT and MRI images in Fig. 2 and Fig. 3 are obtained from the Cancer Imaging Archive (TCIA), specifically from the collections of TCGA-KIRC (72, 73) and LGG-1p19qDeletion (–75), which are publicly and freely available.

Figure 4

Simplified illustration of the model fitting spectrum. Under-fitting (blue dashed-line) and over-fitting (green dashed-line) are common problems to be solved to create more optimally-fitted (red dashed-line) and generalizable models that are useful on unseen or new data. Under-fitting corresponds to the models having poor performance on both training and test data. In general, the under-fitting problem is not discussed because it is evident in the evaluation of performance metrics. Over-fitting, on the other hand, refers to the models having an excellent performance in training data, but very poor performance on test data. In models with over-fitting, the algorithm learns both the relevant data and the noise that is the primary reason of the over-fitting. In reality, all data sets have noise to some extent. However, in case of small data, the effect of the noise could be much more evident. To reduce the over-fitting, possible steps would be to expand the data size, to use data augmentation techniques, to utilize architectures that generalize well, to use regularization techniques (e.g., L1–L2 regularizations and drop-out), and to reduce to the complexity of the architecture or to use less complex classification algorithms. Black and orange circles represent different classes.

Figure 5

Over-simplified illustration of k-nearest neighbors. This machine learning algorithm classifies the unknown objects or instances (blue triangle) by assigning them to the similar objects of the classes (orange vs. black circles) based on the number of neighbors. For instance, considering 3-nearest neighbors, the class represented with black circles outnumbers the other class (orange circles) so that the unknown object is assigned to the class represented with black circles. On the other hand, in case of 5-nearest neighbors, it is assigned to the class with orange circles because the number of the instances in this class outnumbers the one with black circles.

Figure 6

Over-simplified illustration of naive Bayes in a probabilistic space. Naive Bayes is a probabilistic machine learning algorithm and simply based on the strong (naive) independence among the predictor variables (or features). Also, this algorithm assumes that all features equally contribute to the outcome or class prediction. Black and orange circles represent different classes. Black and orange lines represent different probability levels for the instances in different classes.

Figure 7

Over-simplified illustration of logistic regression. Even though many extensions of the logistic regression exist, this algorithm simply uses the logistic function to classify the instances to the binary classes. Black and orange circles represent different classes.

Figure 8

Over-simplified illustrations of support vector machine. In simple terms, this algorithm transforms the original data (left illustration) to a different space (right illustration) to develop optimal plane or vector (red line) that separates the classes. Black and orange circles represent different classes.

Figure 9. a, b

Over-simplified illustrations of decision tree and random forest. In panel (a), decision tree simply creates the most accurate and simple decision points in classification of the instances, providing the most interpretable models for the humans; x, z, and w represent features. In panel (b), to increase the stability and generalizability of the classifications, decision tree algorithm can be iterated several times with various methods. One of the well-known examples is the random forest classifier.

Figure 10

Over-simplified illustration of artificial neural network, particularly deep learning. Neural networks are multi-layer networks of neurons or nodes that are inspired by biological neuronal architecture. Due to computational limitations, early neural networks had very few layers of nodes, generally fewer than 5. Today, it is possible to create useful neural network architectures with many layers. Deep learning or deep neural network generally corresponds to a network with more than 20–25 hidden layers. Variety of deep learning architectures exist in that convolutional neural networks (CNN) are widely used in image analysis. In CNN, image inputs are directly scanned using small-sized filters or kernels, creating transformed images within certain layers like convolutional ones. Convolutional and pooling (or down-sampling) layers are important operations in the CNN architectures, providing the best and most important features of the images (e.g., edges). There are also many important parts of deep learning architectures like activation functions (e.g., rectified linear unit [ReLU], sigmoid function, softmax), regularization (e.g., drop-out layer), and so on. Today, no formula exists to establish the correct number and type of layers for a given classification problem. Therefore, optimal architecture is created with a trial-and-error process. On the other hand, some previously proven architectures and their derivatives are also widely used in similar tasks such as U-Net for segmentation process.

Figure 11

Validation techniques with over-simplified illustrations. In k-fold cross-validation, data set is systematically split to k number of folds, with no overlap in validation parts. In leave-one-out cross-validation, data set is systematically divided to a number that is equal to the number of labeled data set, with no overlap in validation parts. In bootstrapping validation, whole data is sampled to create unseen validation parts that are filled or replaced with similar labeled data in the training data set. In random subsampling, data set is randomly sampled many times to create validation parts that may have overlaps in different experiments. In nested cross-validation, the internal loop is used for feature selection along with model optimization; and external loop is used for model validation to simulate an independent process. In hold-out technique, a single split is created with random sampling. In independent validation, validation part corresponds to a completely different data set, preferably an external data set. Except for bootstrapping validation, black and red circles represent training and validation data sets, respectively.