Basics of Deep Learning: A Radiologist's Guide to Understanding Published Radiology Articles on Deep Learning

Abstract

Artificial intelligence has been applied to many industries, including medicine. Among the various techniques in artificial intelligence, deep learning has attained the highest popularity in medical imaging in recent years. Many articles on deep learning have been published in radiologic journals. However, radiologists may have difficulty in understanding and interpreting these studies because the study methods of deep learning differ from those of traditional radiology. This review article aims to explain the concepts and terms that are frequently used in deep learning radiology articles, facilitating general radiologists' understanding.

Keywords: Artificial intelligence; Convolutional neural network; Deep learning; Radiology.

Fig. 1. Diagram of artificial intelligence hierarchy.

Machine learning is field of study that gives computers ability to learn without being explicitly programmed. Deep learning is subset of machine learning that makes computation of multi-layer neural networks feasible. CNN is subset of deep learning characterized by convolutional layer. CNN = convolutional neural network

Fig. 2. Schematic diagram of CNN model and other regular neural networks.

A. CNN includes convolutional layers, pooling layers, and fully connected layers. B. Each layer is fully connected to all neurons in next layer in other regular neural networks. Conv = convolution

Fig. 3. Convolution and pooling.

A. Example of convolution and pooling. Convolution implies that kernel is applied to input image to produce feature map. Pooling is down-sampling operation. Input, kernel, and feature map are all expressed by matrices, and convolution and pooling are both matrix operations. In this example, a kernel (2 × 2) is applied across input data and element-wise product between kernel and input data is first calculated at each location and then summed to obtain output value in corresponding position of feature map (1 × 1 + 2 × 1 + 5 × 1 + 6 × 1 = 14). In pooling operation, max pooling with filter size of 2 × 2 is applied. Among four values (14, 18, 30, and 34), maximum value (34) is output in corresponding position of next layer. B. Visualization of feature maps through multi-step convolution and pooling. Example of visualized feature maps with chest radiographic image as input for task to differentiate between abdominal and chest radiographs.

Fig. 4. Transfer learning.

Transfer learning is process of taking pre-trained model (usually trained on large dataset, such as ImageNet) and “fine-tuning” model with new dataset. Fully connected layers, with or without parts of kernels of convolutional layers of pre-trained model are replaced with new set and are trained with dataset of new task.

Fig. 5. Model training.

Model training is process of parameter (kernel + weight) optimization through forward and back propagation. Forward propagation is process used to calculate predicted value from input data through model. Back propagation is process used to adjust each parameter of model toward minimizing cost. When presenting series of training samples to model, difference between predicted value and ground truth (target class or regression value) is measured using cost function. Using gradient descent method, all parameters (weights in fully connected layer and kernels in convolutional layer) are slightly adjusted to minimize cost.

Fig. 6. Gradient descent.

Gradient descent is optimization algorithm used to identify parameters that minimize cost. Parameter is repeatedly updated until cost reaches minimum with following formula: W_b = W_a − (learning rate) × (gradient). Optimal parameter of model is value that minimizes cost most. In graph (A), gradient is negative at initial value of parameter (a), and value of (− learning rate × gradient at W_a) is positive. As result, parameter is updated toward increasing value. By repeating this process, minimum of cost function (c) can be obtained, which is parameter's optimal value. On the other hand, gradient is positive at initial value of parameter (a) in graph (B) and value of (− learning rate × gradient at W_a) is negative. As result, parameter is updated in direction of decreasing value, and minimum of cost function (c) is reached. Using gradient descent method, parameter can be optimized regardless of parameter's initial value (initial values of parameters are commonly randomly set). Letter “W” is derived from weight and weight is same as parameter in this case.

Fig. 7. Effect of learning rate on model training.

Learning rate is hyperparameter that determines degree of parameter update. It is important to set learning rate to appropriate value. If learning rate is too small, model will converge too slowly. On the other hand, if learning rate is too large, model will diverge without convergence. Letter “W” is derived from weight and weight is same as parameter in this case.

Fig. 8. Epoch, batch, and iteration.

There is dataset of 6 images. In C, batch size is 2, and algorithm is set to run for 2 epochs. Therefore, in each epoch, there are 3 batches (6 / 2 = 3). Each batch gets passed through algorithm, so there are 3 iterations per epoch. Since 2 epochs were specified, there are total of 6 iterations (3 × 2 = 6) for training. In A, batch size, epoch, and iteration are 3, 1, and 2, respectively. In B, batch size, epoch, and iteration are 2, 1, and 3, respectively.

Fig. 9. Dice score and Jaccard index.

FN = false negative, FP = false positive, TN = true negative, TP = true positive