Transfer learning-based PET/CT three-dimensional convolutional neural network fusion of image and clinical information for prediction of EGFR mutation in lung adenocarcinoma

Abstract

Background: To introduce a three-dimensional convolutional neural network (3D CNN) leveraging transfer learning for fusing PET/CT images and clinical data to predict EGFR mutation status in lung adenocarcinoma (LADC).

Methods: Retrospective data from 516 LADC patients, encompassing preoperative PET/CT images, clinical information, and EGFR mutation status, were divided into training (n = 404) and test sets (n = 112). Several deep learning models were developed utilizing transfer learning, involving CT-only and PET-only models. A dual-stream model fusing PET and CT and a three-stream transfer learning model (TS_TL) integrating clinical data were also developed. Image preprocessing includes semi-automatic segmentation, resampling, and image cropping. Considering the impact of class imbalance, the performance of the model was evaluated using ROC curves and AUC values.

Results: TS_TL model demonstrated promising performance in predicting the EGFR mutation status, with an AUC of 0.883 (95%CI = 0.849-0.917) in the training set and 0.730 (95%CI = 0.629-0.830) in the independent test set. Particularly in advanced LADC, the model achieved an AUC of 0.871 (95%CI = 0.823-0.919) in the training set and 0.760 (95%CI = 0.638-0.881) in the test set. The model identified distinct activation areas in solid or subsolid lesions associated with wild and mutant types. Additionally, the patterns captured by the model were significantly altered by effective tyrosine kinase inhibitors treatment, leading to notable changes in predicted mutation probabilities.

Conclusion: PET/CT deep learning model can act as a tool for predicting EGFR mutation in LADC. Additionally, it offers clinicians insights for treatment decisions through evaluations both before and after treatment.

Keywords: Deep learning; radiomics; Epidermal growth factor receptor; Lung adenocarcinoma; Positron emission tomography/computed tomography.

Fig. 1

Flowchart of patient enrollment. LADC, lung adenocarcinoma; NSCC-NOS, non‑small cell carcinoma‑not otherwise specified

Fig. 2

Overall pipeline for deep learning model development

Fig. 3

ROC curves of different transfer learning models in the training and test sets. AUC, area under the receiver operating characteristic curve; CT_origin, CT model from scratch; CT_TL, CT transfer learning; PET_origin, PET model from scratch; PET_TL, PET transfer learning; DS_TL, dual-stream transfer learning; TS_TL, three-stream transfer learning

Fig. 4

TS_TL predicted tumor-associated areas for solid lesions with either EGFR wild-type or mutation. For each submap, the input CT or PET image, the attention map, and the model-predicted tumor-associated areas are from left to right. For LADC tumors, the deep learning model generated an attention map indicating the importance of each part of the tumor; high-reaction regions (predicted tumor-associated areas) were retained with a cutoff value of 0.5. P⁻ and P⁺ represented the predicted probability of EGFR wild-type and mutant, respectively

Fig. 5

TS_TL predicted tumor-associated areas for subsolid lesions with either EGFR wild-type or mutation. For each submap, the input CT or PET image, the attention map, and the model-predicted tumor-associated areas are from left to right. For LADC tumors, the deep learning model generated an attention map indicating the importance of each part of the tumor; high-reaction regions (predicted tumor-associated areas) were retained with a cutoff value of 0.5. P⁻ and P⁺ represented the predicted probability of EGFR wild-type and mutant, respectively

Fig. 6

The mutant type-associated activation areas of the TS_TL in three EGFR-mutant lesions before and after TKI treatment. Case 1 (stage IV): A female non-smoker in the 60–65 age range presented with a solid mass in the left lower lung (41.1 × 29.7 mm) with an EGFR mutation in exon 20 and received oral poziotinib treatment; through PET/CT re-examination after 21 months, the original lesion shrunk (30.2 × 24.6 mm) and its metabolism was lower than before (SUVmax from 14.8 to 10.2). Poziotinib is a novel targeted drug for rare insertion mutations in exon 20 of EGFR and HER2. Case 2 (stage IV): A male smoker in the 65–70 age range presented with a solid mass in the left upper lung (40.9 × 26.7 mm) harboring an exon 21 EGFR mutation. The patient received oral Osimertinib treatment, and after 6 months, a follow-up PET/CT examination revealed a significant reduction in the size of the original lesion, which became subsolid (20.7 × 15.9 mm), and a decrease in metabolism (SUVmax from 11.8 to 3.9). Case 3 (stage IV): A male non-smoker in the 80–85 age range presented with a subsolid mass in the right upper lung (35.5 × 30.5 mm) carrying an exon 19 EGFR mutation. The patient received oral Icotinib hydrochloride treatment, and after 46 months, a follow-up PET/CT examination revealed a slight reduction in the size of the original lesion (28.5 × 27.9 mm) and a decrease in metabolism (SUVmax from 3.1 to 1.9)