Patient-specific deep learning model to enhance 4D-CBCT image for radiomics analysis

Abstract

Objective.4D-CBCT provides phase-resolved images valuable for radiomics analysis for outcome prediction throughout treatment courses. However, 4D-CBCT suffers from streak artifacts caused by under-sampling, which severely degrades the accuracy of radiomic features. Previously we developed group-patient-trained deep learning methods to enhance the 4D-CBCT quality for radiomics analysis, which was not optimized for individual patients. In this study, a patient-specific model was developed to further improve the accuracy of 4D-CBCT based radiomics analysis for individual patients.Approach.This patient-specific model was trained with intra-patient data. Specifically, patient planning 4D-CT was augmented through image translation, rotation, and deformation to generate 305 CT volumes from 10 volumes to simulate possible patient positions during the onboard image acquisition. 72 projections were simulated from 4D-CT for each phase and were used to reconstruct 4D-CBCT using FDK back-projection algorithm. The patient-specific model was trained using these 305 paired sets of patient-specific 4D-CT and 4D-CBCT data to enhance the 4D-CBCT image to match with 4D-CT images as ground truth. For model testing, 4D-CBCT were simulated from a separate set of 4D-CT scan images acquired from the same patient and were then enhanced by this patient-specific model. Radiomics features were then extracted from the testing 4D-CT, 4D-CBCT, and enhanced 4D-CBCT image sets for comparison. The patient-specific model was tested using 4 lung-SBRT patients' data and compared with the performance of the group-based model. The impact of model dimensionality, region of interest (ROI) selection, and loss function on the model accuracy was also investigated.Main results.Compared with a group-based model, the patient-specific training model further improved the accuracy of radiomic features, especially for features with large errors in the group-based model. For example, the 3D whole-body and ROI loss-based patient-specific model reduces the errors of the first-order median feature by 83.67%, the wavelet LLL feature maximum by 91.98%, and the wavelet HLL skewness feature by 15.0% on average for the four patients tested. In addition, the patient-specific models with different dimensionality (2D versus 3D) or loss functions (L1 versus L1 + VGG + GAN) achieved comparable results for improving the radiomics accuracy. Using whole-body or whole-body+ROI L1 loss for the model achieved better results than using the ROI L1 loss alone as the loss function.Significance.This study demonstrated that the patient-specific model is more effective than the group-based model on improving the accuracy of the 4D-CBCT radiomic features analysis, which could potentially improve the precision for outcome prediction in radiotherapy.

Keywords: 4D-CBCT; CBCT; deep learning; generative adversarial network; patient-specific; radiomics; under-sampled projections.

Figure 1

Workflow of the entire study

Figure 2

Architecture of the pix2pix model. The yellow block represents the generator and it utilizes the architecture of Resnet 9 blocks. The large blue block represents the architecture of the discriminator.

Figure 3

Tumor locations of each patient shown by MITK software (Goch et al., 2017)

Figure 4

4D-CBCT images before and after enhancement with W/L = 2200/100. Red arrows show the same locations of different images, which could help detail comparisons across different images. The red circles highlight the region of interest. The difference between ground truth CT and each image is also plotted. (A) The ground truth 4D-CT images. (B) The original under-sampled 4D-CBCT before enhancement. (C) The enhanced 4D-CBCT by the group-based model. D-G are results from patient-specific models: (D) The enhanced 4D-CBCT image using 2D pix2pix model with only whole image L1 loss. (E) The enhanced 4D-CBCT image using 3D pix2pix model with only whole image L1 loss. (F) The enhanced 4D-CBCT image using 3D pix2pix model with both whole image L1 and ROI L1 loss functions. (G) The enhanced 4D-CBCT image using 3D pix2pix model with whole image L1, ROI L1, VGG, and GAN loss functions. (H-M) are the corresponding difference images between ground truth CT and (B-G). The color bar on the right shows the difference of each pixel value between the enhancement results and ground truth CT.

Figure 4

4D-CBCT images before and after enhancement with W/L = 2200/100. Red arrows show the same locations of different images, which could help detail comparisons across different images. The red circles highlight the region of interest. The difference between ground truth CT and each image is also plotted. (A) The ground truth 4D-CT images. (B) The original under-sampled 4D-CBCT before enhancement. (C) The enhanced 4D-CBCT by the group-based model. D-G are results from patient-specific models: (D) The enhanced 4D-CBCT image using 2D pix2pix model with only whole image L1 loss. (E) The enhanced 4D-CBCT image using 3D pix2pix model with only whole image L1 loss. (F) The enhanced 4D-CBCT image using 3D pix2pix model with both whole image L1 and ROI L1 loss functions. (G) The enhanced 4D-CBCT image using 3D pix2pix model with whole image L1, ROI L1, VGG, and GAN loss functions. (H-M) are the corresponding difference images between ground truth CT and (B-G). The color bar on the right shows the difference of each pixel value between the enhancement results and ground truth CT.

Figure 5

Comparison of feature errors in original and enhanced CBCTs. Feature 1–10 are: 1. median, 2. GLCM cluster shade, 3. LLH range, 4. LHL total energy, 5. HLL skewness, 6. LLL max, 7. LoG 3mm skewness, 8. LoG 5mm skewness, 9. LoG 3mm GLCM inverse difference, and 10. LoG 5mm GLRLM short-run emphasis.

Figure 6

Comparison of errors of low-level features in the original CBCT and CBCT enhanced by group-based and patient-specific models in patient 3.

Figure 7

Comparison of 2D and 3D models for reducing the radiomic feature errors.

Figure 8

Comparison of the impact of ROI loss functions for reducing the radiomic feature errors.

Figure 9

Comparison of L1, VGG, and GAN loss functions for reducing the radiomic feature errors.