Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept

Abstract

[¹⁸F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Keywords: Turing test; generative AI; generative adversarial network; image synthesis; lung cancer; prognostic marker; radiogenomics; radiomics; screening; synthetic PET.

Graphical abstract

Figure 1

Overview of the study design (A) Training the cGAN to predict PET image from CT. The input of the generator is a CT slice along with its six neighboring slices while its output is the synthetic PET image. The discriminator tries to classify between the synthetic and ground-truth PET/CT pairs. (B) In the imaging validation, similarity metrics including SSIM and RMSE were employed for comparing the synthetic and ground-truth PET images. A group of two thoracic radiologists was enrolled blindly to visually assess the quality of synthetic PET images. Next, they conducted a Turing test on synthetic and ground-truth PET images. Moreover, we analyzed the pairwise similarity between the synthetic and ground-truth PET features. (C) In the biological validation, we applied radiogenomic analysis using the GSEA method to find the association of cancer hallmarks with extracted features from ground-truth and synthetic PET scans. (D) In the subsection “synthetic PET complements CT for early lung cancer diagnosis,” we investigated whether the performance of indeterminate pulmonary nodule classification using only main CT-based features can be further improved by adding the extracted features from the synthetic PET. In cancer prediction, we validated the clinical value of synthetic PET in prediction of the development of lung cancer and in staging prediction. In staging prediction, two radiologists assess the synthetic PET capability for accurately staging the lung cancer patients. In the subsection “synthetic PET predicts prognosis after standard of care,” we showed that the extracted nodule features from the synthetic PET are capable of stratifying patients into good and bad survival groups. (E) The cohorts used in different sections.

Figure 2

Validation of imaging signal fidelity and cancer staging performance by radiologists (A) Presentation of synthetic PET images with ground-truth PET in MDA-TEST and TCIA-STANFORD testing cohorts. The first three columns from left to right for each cohort correspond to CT, ground-truth, and synthetic PET images. The PET images are shown inversely with the normalized window of SUV in $\left[0,3\right]$. Therefore, the completely black color in tumors indicates that the tumor had uptake with maximum SUV value of at least 3. (B) The radiologists’ score on quality and relative uptake of lung region in task 1 of the imaging quality test. (C) Alluvial plot shows the radiologists’ scoring on imaging quality difference using paired PET scans. (D) Alluvial plot shows the radiologists’ scoring on tumor contrast difference using paired PET scans. (E) Alluvial plot shows the radiologists’ reading of ground-truth vs. synthetic using paired PET scans. (F) Comparison matrix of staging between radiologists reading CT and ground-truth PET and pathological stage. (G) Comparison matrix of staging between radiologists reading CT and synthetic PET and pathological stage. (H) Consensus matrix of staging between the two radiologists when one radiologist reads true PET and CT compared to another reading synthetic PET and CT.

Figure 3

Validation of imaging signal fidelity by PET feature correlation (A) Pearson correlation for evaluating the pairwise similarity between the synthetic (rows) and ground-truth (columns) PET features in MDA-TRAIN, MDA-TEST, and TCIA-STANFORD cohorts. (B) Threshold-based confusion matrix for synthetic SUV_max and the ground-truth SUV_max for four different thresholds ($\alpha =1.5,2.5$) in MDA-TRAIN, MDA-TEST, and TCIA-STANFORD cohorts.

Figure 4

Validation of biological fidelity by radiogenomics analysis (A) Significant hallmark gene sets associated with MTV extracted from ground-truth and synthetic PET for MDA-TRAIN, TCIA-STANFORD, and MDA-TEST cohorts. (B) The first column shows the unsupervised hierarchical clustering heatmap of hallmark pathways normalized enrichment score for correlation of each pathway with MTV feature from ground-truth and synthetic PET across MDA-TRAIN, TCIA-STANFORD, and MDA-TEST cohorts, where the asterisks represent the significant false discovery rate q value >0.25.

Figure 5

Validation of clinical value by diagnosing malignant vs. benign from indeterminate pulmonary nodules (A) Model accuracy in the test cohort (n = 350) obtained from synthetic PET univariate features (MTV, SUV_max), CT univariate features (tumor_size, tumor_max-d), and CT and PET bivariate features (tumor_size & MTV, tumor_size & SUV_max, tumor_max-d & MTV, tumor_max-d & SUV_max) in the LIDC-IDRI cohort. (B) The first column corresponds to the confusion matrix of the best performance obtained using either of single CT features tumor_size or tumor_max-d. The second column corresponds to the confusion matrix of the best performance when adding one synthetic PET feature, SUV_max or MTV. (C) Threshold-based confusion matrix for synthetic SUV_max and the nodule malignancy at four different thresholds ($\alpha =1.5,2.5$) in the LIDC-IDRI cohort.

Figure 6

Validation of clinical value of synthetic PET by predicting the risk of developing lung cancer (A) Distribution of CT scans in training, validation, and test sets in both group 1 and group 2. (B) Two box plots were used to compare the synthetic SUV_max values in group 1 (lung cancer diagnosed within 1 year vs. lung cancer diagnosed at >3 years) and group 2 (lung cancer diagnosed within 1 year vs. lung cancer diagnosed at >5 years). (C) Receiver-operating characteristic curves for CT and CT plus synthetic PET analysis along with their respective AUC values for both group 1 and group 2.

Figure 7

Validation of clinical value of synthetic PET by predicting overall survival Kaplan-Meier curves of patients’ overall survival (OS) stratified by MTV_1.5 and MTV_2.5 features obtained from the synthetic PET images on MDA cohorts (MDA-TRAIN and MDA-TEST) and non-MDA cohorts (TCIA-STANFORD and NSCLC-RT).