Prediction of prognosis in glioblastoma with radiomics features extracted by synthetic MRI images using cycle-consistent GAN

Abstract

To propose a style transfer model for multi-contrast magnetic resonance imaging (MRI) images with a cycle-consistent generative adversarial network (CycleGAN) and evaluate the image quality and prognosis prediction performance for glioblastoma (GBM) patients from the extracted radiomics features. Style transfer models of T1 weighted MRI image (T1w) to T2 weighted MRI image (T2w) and T2w to T1w with CycleGAN were constructed using the BraTS dataset. The style transfer model was validated with the Cancer Genome Atlas Glioblastoma Multiforme (TCGA-GBM) dataset. Moreover, imaging features were extracted from real and synthesized images. These features were transformed to rad-scores by the least absolute shrinkage and selection operator (LASSO)-Cox regression. The prognosis performance was estimated by the Kaplan-Meier method. For the accuracy of the image quality of the real and synthesized MRI images, the MI, RMSE, PSNR, and SSIM were 0.991 ± 2.10 $\times {10}^{-4}$ , 2.79 ± 0.16, 40.16 ± 0.38, and 0.995 ± 2.11 $\times {10}^{-4}$ , for T2w, and .992 ± 2.63 $\times {10}^{-4}$ , 2.49 ± 6.89 $\times {10}^{-2}$ , 40.51 ± 0.22, and 0.993 ± 3.40 $\times {10}^{-4}$ for T1w, respectively. The survival time had a significant difference between good and poor prognosis groups for both real and synthesized T2w (p < 0.05). However, the survival time had no significant difference between good and poor prognosis groups for both real and synthesized T1w. On the other hand, there was no significant difference between the real and synthesized T2w in both good and poor prognoses. The results of T1w were similar in the point that there was no significant difference between the real and synthesized T1w. It was found that the synthesized image could be used for prognosis prediction. The proposed prognostic model using CycleGAN could reduce the cost and time of image scanning, leading to a promotion to build the patient's outcome prediction with multi-contrast images.

Keywords: Cycle-GAN; GAN; Machine learning; Neural network; Radiomics.

Fig. 1

Overview of the process of Radiomics-based prognosis prediction models using style transfer. The prognosis prediction model constructed consists of the following processes: Style transfer, Segmentation, Feature Extraction, Dimension Reduction, Prognosis prediction. Dimension Reduction is further composed of parts Normality check, Pearson Test, and LASSO Cox regression. The upper part of the figure shows Real image, and the lower part shows the flow of the synthesized image

Fig. 2

Training dataset for style transfer of (a) T1w, (b) T2w, and (c) tumor segmentation

Fig. 3

Flowchart of patient exclusion for (a) style transfer dataset and (b) Radiomics dataset using BraTS-TCGA-GBM

Fig. 4

T1w to T2w and T2w to T1w translation networks using CycleGAN. rT1w and rT2w are real image sT2w and sT1w are synthesized image. G_T1→T2 and G_T2→T1 are the generator. The generator produces a synthesized image by transforming the contrast of the image. D_{T1, sT1} and D_{T2, sT2} are the discriminator. The discriminator distinguishes between real image and synthesized images

Fig. 5

An architecture for the CycleGAN generator. The representation size shrinks in the encoder phase, stays constant in the transformer phase, and expands again in the decoder phase. The representation size that each layer outputs is listed below it, in terms of the input image size, K. On each layer is listed the number of channels. Each layer is followed by an instance normalization and Rectified Linear Unit (ReLU) activation function

Fig. 6

An architecture for the CycleGAN discriminator. It calls PatchGAN to discriminate real and synthesized images for each region cut out by the patch size. The representation size from each layer is listed below each layer in terms of the input image size, K. On each layer is listed the number of channels

Fig. 7

Example of a style transfer testing: (a) rT2w, (b) sT2w, and (c) difference image between rT2w and sT2w. The color bar of the difference image was set to the maximum value of the difference image

Fig. 8

Example of a style transfer testing: (a) rT1w, (b) sT1w, and (c) difference image between rT1w and sT1w. The color bar of the difference image was set to the maximum value of the difference image

Fig. 9

LASSO-Cox regression selected features (a) rT2w and (b) sT2w. The features selected in (a) and (b), respectively, have coefficients obtained by Lasso-cox regression. These coefficients are used to construct the Rad-score

Fig. 10

LASSO-Cox regression selected features (a) rT1w and (b) sT1w. The features selected in (a) and (b), respectively, have coefficients obtained by Lasso-cox regression. These coefficients are used to construct the Rad-score

Fig. 11

The comparison of survival curves between the good prognosis group and poor prognosis group using (a) rT2w-rad-score and (b) sT2w-rad-score, respectively

Fig. 12

The comparison of survival curves of the rT2w-rad-score and sT2w-rad-score in (a) good prognosis and (b) poor prognosis

Fig. 13

The comparison of survival curves in the good prognosis group and poor prognosis group using rT1w-rad-score (a) and sT1w-rad-score (b), respectively

Fig. 14

The comparison of survival curves of the rT1w-rad-score and sT1w-rad-score in (a) good prognosis and (b) poor prognosis

Fig. 15

An example of image processing by LoG filter (a) original; (b) $\sigma$ = 1 mm; (c) $\sigma$ = 2 mm; (d) $\sigma$ = 3 mm; (e) $\sigma$ = 4 mm; and (f) $\sigma$ = 5 mm. $\sigma$ is standard deviation of pixel values

Fig. 16

An example of image processing by 2D-wavelet transform (a) Approximation; (b) horizontal detail; (c) vertical detail; (d) diagonal detail. The color bar in the figure was set by the maximum value of pixel