Radiological tumor classification across imaging modality and histology

Abstract

Radiomics refers to the high-throughput extraction of quantitative features from radiological scans and is widely used to search for imaging biomarkers for prediction of clinical outcomes. Current radiomic signatures suffer from limited reproducibility and generalizability, because most features are dependent on imaging modality and tumor histology, making them sensitive to variations in scan protocol. Here, we propose novel radiological features that are specially designed to ensure compatibility across diverse tissues and imaging contrast. These features provide systematic characterization of tumor morphology and spatial heterogeneity. In an international multi-institution study of 1,682 patients, we discover and validate four unifying imaging subtypes across three malignancies and two major imaging modalities. These tumor subtypes demonstrate distinct molecular characteristics and prognoses after conventional therapies. In advanced lung cancer treated with immunotherapy, one subtype is associated with improved survival and increased tumor-infiltrating lymphocytes compared with the others. Deep learning enables automatic tumor segmentation and reproducible subtype identification, which can facilitate practical implementation. The unifying radiological tumor classification may inform prognosis and treatment response for precision medicine.

Figure 1.

Morphological characterization of tumors by spherical harmonic decomposition. a) Overall design of morphological analysis; b) Illustration of 3D spherical harmonic basis functions at different degrees and orders; c) Illustration of 3D tumors reconstructed by coefficients obtained from spherical harmonic decomposition. Each row represents a selected 3D tumor, which is reconstructed using decomposition results at 5 different degree levels. Here, lower degree captures more global patterns and higher degree corresponds to more detailed morphological patterns.

Figure 2.

The ridgeline plots present the distribution of 20 regional variation features in three different cancer types. Here, we investigate 2 tumor regions, tumor core (TC) and tumor invasive margin (TIM), plus 2 peritumor regions, parenchymal margin at 5 mm or 10 mm (PM5 or PM10). In total, 5 pair-wise regions are considered, namely, TC-TIM, TC-PM5, TC-PM10, TIM-PM5, TIM-PM10. Variation for each pair-wise region was quantified with four measures (chi-square, Bhattacharyya distance, correlation, intersection), yielding 5*4 = 20 regional variation features. TC-PM5 and TC-PM10 related features are colored in green, while TIM-PM5 and TIM-PM10 related features are colored in blue.

Figure 3.

Details of imaging feature dimension reduction via an autoencoder model. a) The structure of autoencoder used to learn a low-dimensional mapping of the original feature signals with detailed tuning hyperparameters; b) The optimal autoencoder loss curves in training and validation; c) Heatmap of pairwise correlations between 10 autoencoded features.

Figure 4.

Distribution of imaging clusters (subtypes) in different clinical groups. a) The distribution of all patients in four clusters (subtypes) across three cancer types; b) The distribution of lung cancer patients in four clusters (subtypes) across different clinical stage; The molecular subtype distribution in four imaging subtypes for c) breast cancer with luminal A/B, Her2+, and triple negative; d) GBM with different MGMT methylation status.

Figure 5.

Volcano plot of enrichment scores through single-sample Gene Set Enrichment Analysis (ssGSEA) of 313 proposed imaging features in all three cancer types. a) imaging subtype 1 versus rest, b) subtype 2 versus rest, c) subtype 3 versus rest, and d) subtype 4 versus rest. The data for all enrichment scores are plotted as log2 fold change versus the −log10 of the adjusted p-value. Thresholds are shown as dashed lines. Pathways deemed as significantly different (false discovery rate or FDR<0.05) are highlighted with different color schemes.

Figure 6.

Evaluation of prognostic value of the four imaging subtypes in lung cancer subgroups. Kaplan-Meier curves for a) stage I+II; b) Stage III; c) Patients treated with surgery; d) Patients treated with radiation.

Figure 7.

Evaluation of prognostic value of the four imaging subtypes in subgroups within three cancer types. Kaplan-Meier curves for lung cancer subgroups: a) EGFR Wild Type; b) EGFR Mutant; c) ALK Wild Type; for breast cancer subgroups: d) ER+ group; e) HER2+ group; f) Triple Negative (TN) group; for GBM cancer subgroups: g) MGMT Methylated group; h) MGMT Unmethylated group; i) IDH1 Wild group.

Figure 8.

Comparison between the proposed imaging subtypes and conventional radiomics analysis for survival prediction in lung cancer cohorts. a) Details of the final radiomic model; b) Distribution of the radiomic risk score in training and validation cohorts; c) Scatterplot shows the correlation between radiomic risk score and tumor size measured in 2D; d) Distribution and comparison of c-index for the radiomic signature and the proposed imaging subtypes in the validation cohort.

Figure 9.

Oncogenic processes associated with the imaging subtypes in three cancer types. Limma-modeled enrichment analysis by single-sample Gene Set Enrichment Analysis (ssGSEA) of 50 cancer hallmark pathways is applied. Volcano plot of enrichment scores in lung cancer: a) subtype 1 versus rest, and b) subtype 4 versus rest; in breast cancer: c) subtype 1 versus rest, and d) subtype 4 versus rest; in GBM: e) subtype 1 versus rest, and f) subtype 4 versus rest. The enrichment scores of 50 cancer hallmark pathways are plotted as log2 fold change versus the −log10 of the adjusted p-value. Thresholds are shown as dashed lines. Pathways deemed as significantly different (false discovery rate [FDR] < 0.05) are highlighted with different color schemes.

Figure 10.

Evaluation of imaging subtypes in the advanced lung cancer treated with immunotherapy. Kaplan-Meier curves of overall survival stratified by imaging subtype 1 and 2 versus 4.

Figure 1.

Overview of the study design and quantitative imaging analysis. a) Study design, which contains five phases; b-d) Illustration of the proposed image feature extraction pipeline. First, the primary tumor was manually delineated and surrounding parenchymal tissues (i.e., lung, fibro-glandular, and brain) were automatically segmented, b). Then, two broad categories of image features were calculated, including c), systematic shape descriptors through spherical harmonic decomposition and d) spatial heterogeneity described by regional variations among tumor core, tumor invasive margin, and parenchymal region.

Figure 2.

Identification of unifying tumor subtypes based on unsupervised consensus clustering of the extracted image features across three cancer types and across two modalities (CT and MRI). The consensus matrix corresponding to the optimal cluster number (k=4) for a) discovery set, b) validation set, c) CT set, d) MRI set, and e) whole population. Patients are both rows and columns. The matrix is ordered by consensus-clustered groups, depicted as a dendrogram above the heat map. f) The cluster purity score of four tumor subtypes in three individual cancer types.

Figure 3.

Radiological characteristics of the unifying tumor subtypes. a) Heatmap of four subtypes with respect to the original imaging features; b) Boxplots of four representative groups of features including tumor volume, positive-correlated regional variation, shape symmetry, and shape irregularity, stratified by imaging subtypes as well as cancer types; c) Summary of key imaging characteristics of four subtypes; d) Schematic diagram for distribution of the imaging subtypes in a 3D space formed by tumor size, shape complexity, and regional variation.

Figure 4.

Evaluation of prognostic value of the four imaging subtypes in three individual cancer types. Kaplan-Meier curves for a) overall survival in lung cancer, b) recurrence-free survival in breast cancer, and c) overall survival in glioblastoma multiforme. d-f) Forest plots show the hazard ratio and p values obtained from a multivariate Cox regression analysis including the proposed imaging subtypes and established clinicopathologic factors in different cancer types.

Figure 5.

Clinical evaluation of the imaging subtypes in advanced lung cancer treated with immunotherapy. a) Distribution of inferred imaging subtypes; b) Representative features stratified by imaging subtypes, including tumor volume, positive-correlated regional variation, shape symmetry, and shape irregularity; c) Kaplan-Meier curves of overall survival stratified by imaging subtype 3 versus 4; d) Comparison of tumor-infiltrating immune cell populations between subtypes 4 and 3.

Figure 6.

Deep learning to automate 3D tumor segmentation. a) The detailed architecture of proposed U-Net with dilation at bottleneck layers; b) DICE coefficients of trained U-Net model in NSCLC in training, validation, and testing sets; DICE coefficients of U-Net stratified by c) tumor sizes, and d) anatomical locations of the tumor; e) Representative CT slices and tumor contours of four different testing patients; f) Confusion matrix of clustering results based on manual tumor contouring and automated segmentation for NSCLC patients.