Introduction to Machine Learning, Neural Networks, and Deep Learning

Abstract

Purpose: To present an overview of current machine learning methods and their use in medical research, focusing on select machine learning techniques, best practices, and deep learning.

Methods: A systematic literature search in PubMed was performed for articles pertinent to the topic of artificial intelligence methods used in medicine with an emphasis on ophthalmology.

Results: A review of machine learning and deep learning methodology for the audience without an extensive technical computer programming background.

Conclusions: Artificial intelligence has a promising future in medicine; however, many challenges remain.

Translational relevance: The aim of this review article is to provide the nontechnical readers a layman's explanation of the machine learning methods being used in medicine today. The goal is to provide the reader a better understanding of the potential and challenges of artificial intelligence within the field of medicine.

Keywords: artificial intelligence; deep learning; machine learning.

Figure 1.

Umbrella of select data science techniques. Artificial intelligence (AI) falls within the realm of data science, and includes classical programming and machine learning (ML). ML contains many models and methods, including deep learning (DL) and artificial neural networks (ANN).

Figure 2.

Classical programming versus machine learning paradigm. (A) In classical programming, a computer is supplied with a dataset and an algorithm. The algorithm informs the computer how to operate upon the dataset to create outputs. (B) In machine learning, a computer is supplied with a dataset and associated outputs. The computer learns and generates an algorithm that describes the relationship between the two. This algorithm can be used for inference on future datasets.

Figure 3.

Sensitivity, specificity, positive predictive value, and negative predictive value. A population (dataset) is represented as circles colored blue if positive or orange if negative. The dataset is input to an algorithm that predicts each instance's class association. If an instance is correctly predicted as positive or negative, it is a true positive (TP) or true negative (TN), respectively. If an instance is incorrectly labeled positive or negative, it is a false positive (FP) or false negative (FN), respectively. (A) A model with perfect sensitivity ($\sum \frac{TP}{TP+FN}$ ) and specificity ($\sum \frac{TN}{TN+FP}$). (B) A model with perfect sensitivity (ability to correctly classify all positive cases), but poor specificity (ability to correctly classify all negative cases) and (C) a model with perfect specificity, but poor sensitivity. Although a model might have perfect sensitivity (B), it can have many false positives. Similarly, a model with perfect specificity (C) might have many false negatives. Therefore, it is also useful to evaluate the positive predictive value (PPV; $\sum \frac{TP}{TP+FP}$ ) and the negative predictive value (NPV; $\sum \frac{TN}{TN+FN}$). PPV and NPV are also thus dependent on the prevalence of disease in a population.

Figure 4.

Example receiver operating characteristics and precision-recall curves. Red line: a model that performs no better than chance has an area under the curve (AUC) of the receiver operating characteristics curve (AUROC) of 0.50 or area under the precision-recall curve (AUPR) at the class ratio (red shaded area). Blue line: a model that performs better than chance, but not perfectly, will have an AUC between 0.50 and 1.00 (blue + red shaded areas). Green line: a model that performs perfectly has an AUC of 1.00 (red + blue + green shaded areas).

Figure 5.

Example class probability prediction using linear and logistic regression. Presented are linear (blue line) and logistic (red line) regression models for predicting the probability of various samples (gray circles) as belonging to a particular class using a single variable, variable X, which ranges from -10 to 10. With logistic regression, variable X is transformed into class probabilities that are bounded between 0 and 1 using the sigmoid function. Simple linear regression attempts to estimate class probabilities, but is not bounded between 0 and 1; thus, it breaks a fundamental law of probability that does not allow for negative probabilities or those greater than 1.

Figure 6.

Structure of a decision tree. Splitting of the dataset begins at the root node. Each split connects to either another decision node, which results in further splitting of the data, or a terminal node that predicts the class of the data.

Figure 7.

Components of a neural network. (A) The basis of an artificial neural network, the perceptron. This algorithm uses the sigmoid function to scale and transform multiple inputs into a single output ranging from 0 to 1. (B) An artificial neural network connects multiple perceptron units, so that the output of one unit is used as input to another. Additionally, these units are not limited to using the sigmoid activation function. (C) Examples of four different activation functions: sigmoid, hyperbolic tangent, identity, and rectified linear unit. The sigmoid scales inputs between 0 and 1 using an S-shaped curved. Similarly, the hyperbolic tangent function uses an S-shaped curve, but scales inputs between -1 and 1. The identity function can multiply its input by any number to produce a linear output. The rectified linear unit is similar to the identity function, however all inputs < 0 are given an output value of 0. There are other activation functions outside of these, but these are arguably.

Figure 8.

Example of a digital image convolved with a filter. The image (left) is transformed into the feature map (right) via a convolutional filter (center). The convolutional filter is designed to locate diagonal lines running from top left to bottom right of the image. The filter passes over the image in a specified manner and each element in the image (red) is multiplied by the corresponding element in the convolutional filter (blue). The summation of these elements (orange) is output into a new matrix that reports the presence of a diagonal line. The feature map indicates 2 when the specified diagonal line is found, 1 if a portion of it is found, and 0 if none of it is found.