Radiomics: from qualitative to quantitative imaging

Abstract

Historically, medical imaging has been a qualitative or semi-quantitative modality. It is difficult to quantify what can be seen in an image, and to turn it into valuable predictive outcomes. As a result of advances in both computational hardware and machine learning algorithms, computers are making great strides in obtaining quantitative information from imaging and correlating it with outcomes. Radiomics, in its two forms "handcrafted and deep," is an emerging field that translates medical images into quantitative data to yield biological information and enable radiologic phenotypic profiling for diagnosis, theragnosis, decision support, and monitoring. Handcrafted radiomics is a multistage process in which features based on shape, pixel intensities, and texture are extracted from radiographs. Within this review, we describe the steps: starting with quantitative imaging data, how it can be extracted, how to correlate it with clinical and biological outcomes, resulting in models that can be used to make predictions, such as survival, or for detection and classification used in diagnostics. The application of deep learning, the second arm of radiomics, and its place in the radiomics workflow is discussed, along with its advantages and disadvantages. To better illustrate the technologies being used, we provide real-world clinical applications of radiomics in oncology, showcasing research on the applications of radiomics, as well as covering its limitations and its future direction.

Figure 1.

Timeline highlighting key developments in medical imaging. CAD, computer-aided diagnosis; GLCM, grey level co-occurring matrix; PET, positron emission tomography.

Figure 2.

The difference between using (A) a contoured binary mask, and (B) using a bounding box.

Figure 3.

Possible angles for the calculation of co-occurrence matrices in two and three dimensions. (A) Shows the 4 possible directions in 2 dimensions while (B) shows the 13 possible directions in 3 dimensions.

Figure 4.

Calculating a GLCM for horizontal co-occurring pixel intensities. In total, 3 co-occurring pixel intensities of 3 and 2 that are next to each other on a horizontal plane can be totalled and tracked in the corresponding matrix. GLCM, grey level co-occurring matrix.

Figure 5.

An example of fivefold cross-validation which can be used to evaluate machine learning models. Cross-validation gives the ability to test the result across the entirety of a data set, giving a better estimation of a model’s overall performance.

Figure 6.

The architecture of a single neuron with a transfer function and a sigmoid activation function visualised.

Figure 7.

A three layer neural network that is a binary classifier with three inputs. Nodes with X_n refer to inputs while other nodes refer to activation functions. The connecting lines between the nodes represent weights.

Figure 8.

A filter that is able to filter out vertical lines. The yellow lines represent the kernel or sliding window, while the image on the right is the result of performing convolutions across the entirety of the original image.