The transformational potential of molecular radiomics

Abstract

Conventional radiomics in nuclear medicine involve hand-crafted and computer-assisted regions of interest. Recent developments in artificial intelligence (AI) have seen the emergence of AI-augmented segmentation and extraction of lower order traditional radiomic features. Deep learning (DL) affords the opportunity to extract abstract radiomic features directly from input tensors (images) without the need for segmentation. These fourth-order, high dimensional radiomics produce deep radiomics and are well suited to the data density associated with the molecular nature of hybrid imaging. Molecular radiomics and deep molecular radiomics provide insights beyond images and quantitation typical of semantic reporting. While the application of molecular radiomics using hand-crafted and computer-generated features is integrated into decision-making in nuclear medicine, the acceptance of deep molecular radiomics is less universal. This manuscript aims to provide an understanding of the language and principles associated with radiomics and deep radiomics in nuclear medicine.

Keywords: Artificial intelligence; artificial neural network; convolutional neural network; deep learning; deep radiomics; molecular imaging; nuclear medicine; radiomics.

Figure 1

Hand‐crafted (blue) and computer‐assisted using thresholding (yellow and pink) ROIs on transverse PET (A) and PET/CT (B), and on sagittal PET (C) and PET/CT (D). The ROIs are used to produce a raft of radiomics including an SUV_mean of 2.5. Image reproduced courtesy of Cegla et al 2022.

Figure 2

AI‐augmented radiomic feature extraction for myocardial perfusion (top) using polar maps and summed defect scores, and myocardial function (bottom) using polar maps (motion, EF and thickening) and parameters (LVEF and volumes).

Figure 3

Flow chart highlighting the semantic pathway for image quantitation and reporting (right). Traditional radiomic feature analysis (middle) and deep radiomic feature analysis (left) also have pathways outlined.

Figure 4

Radiomic feature extraction pipeline from imaging (green arrows) through segmentation and first‐, second‐ and third‐order feature extraction, analysis and classification (red arrows) to reporting (purple arrows). Semantic reporting may bypass radiomic analysis and classifiers. The potential applications of deep neural networks to produce more abstract fourth‐order deep radiomic features is also represented (orange arrows). The schematic differentiates deep learning approaches to image segmentation for first‐, second‐ and third‐order radiomic feature generation from the direct extraction of abstract features (fourth‐order) from the deep layers of a neural network.

Figure 5

DL architecture to produce fourth‐order radiomic features from thyroid images. Image reprinted with permission Currie and Iqbal 2022.

Figure 6

Radiomic feature matrix with the depth of insight represented on the vertical axis, the degree of automation represented on the horizontal axis and the radiomic dimensionality indicated by the red diagonal arrow. Increasing dimensionality is accompanied by increasing cost and decreasing accessibility. Decreased accessibility to high dimensionality tools could drive social asymmetry. Universality is generally associated with the lower dimensionality, and this corresponds to widespread clinical application of these radiomic features (green zone). The blue zone represents emerging tools and technology with clinical application; albeit limited. The orange zone represents the cutting edge of AI technology and, with research, may emerge as future clinical applications.

Figure 7

Schematic representation of the DL approach to detection, segmentation, classification and lower order radiomic feature generation (top). Schematic representation of the DL approach to classification from fourth‐order radiomic feature generation (bottom). Image modified and reprinted with permission Currie 2022.