Generative Adversarial Networks Can Create High Quality Artificial Prostate Cancer Magnetic Resonance Images

Abstract

The recent integration of open-source data with machine learning models, especially in the medical field, has opened new doors to studying disease progression and/or regression. However, the ability to use medical data for machine learning approaches is limited by the specificity of data for a particular medical condition. In this context, the most recent technologies, like generative adversarial networks (GANs), are being looked upon as a potential way to generate high-quality synthetic data that preserve the clinical variability of a condition. However, despite some success, GAN model usage remains largely minimal when depicting the heterogeneity of a disease such as prostate cancer. Previous studies from our group members have focused on automating the quantitative multi-parametric magnetic resonance imaging (mpMRI) using habitat risk scoring (HRS) maps on the prostate cancer patients in the BLaStM trial. In the current study, we aimed to use the images from the BLaStM trial and other sources to train the GAN models, generate synthetic images, and validate their quality. In this context, we used T2-weighted prostate MRI images as training data for Single Natural Image GANs (SinGANs) to make a generative model. A deep learning semantic segmentation pipeline trained the model to segment the prostate boundary on 2D MRI slices. Synthetic images with a high-level segmentation boundary of the prostate were filtered and used in the quality control assessment by participating scientists with varying degrees of experience (more than ten years, one year, or no experience) to work with MRI images. Results showed that the most experienced participating group correctly identified conventional vs. synthetic images with 67% accuracy, the group with one year of experience correctly identified the images with 58% accuracy, and the group with no prior experience reached 50% accuracy. Nearly half (47%) of the synthetic images were mistakenly evaluated as conventional. Interestingly, in a blinded quality assessment, a board-certified radiologist did not significantly differentiate between conventional and synthetic images in the context of the mean quality of synthetic and conventional images. Furthermore, to validate the usability of the generated synthetic images from prostate cancer MRIs, we subjected these to anomaly detection along with the original images. Importantly, the success rate of anomaly detection for quality control-approved synthetic data in phase one corresponded to that of the conventional images. In sum, this study shows promise that high-quality synthetic images from MRIs can be generated using GANs. Such an AI model may contribute significantly to various clinical applications which involve supervised machine-learning approaches.

Keywords: MRI; generative adversarial networks; image segmentation; machine learning.

Figure 1

Image generation using SinGAN. SinGAN trains a model using generative adversarial networks in a course-to-fine manner. At each scale, the generator makes sample images that cannot be distinguished in down-sampled training images by the discriminator. The 8th, 9th, and 10th scale images (the three rightmost images) of the model were resized and used in all quality control tests.

Figure 2

(A) Example of training data for the image segmentation neural network and the desired result. The top left is the T2 tse training image, and the bottom left is the corresponding binary contour. (B) The deep learning image segmentation training progression is shown for one image of the training set. The final predicted contour had a dice similarity coefficient of 0.99 to the real contour. (C) To pass the deep learning segmentation pipeline, the predicted contour had to be one continuous contour greater than 10,000 pixels. (D) Examples of the 253 synthetic images that passed the deep learning segmentation pipeline, shown with the prediction boundary.

Figure 3

Boxplots depicting the minimum, median, interquartile range, maximum and potential outliers of image intensity for both synthetic and conventional samples. Highlighting the convolutional neural network quality control and applicability validation steps in evaluating the distribution of intensity for synthetic and conventional images, mean intensity for synthetic and conventional images, and distribution of synthetic image intensity by cancer stage. Here “N” represents normal and 1–5 represents the increasing progression of the disease.

Figure 4

Workflow of procedure for second human quality control assessment. Three synthetic samples output by the last three generative models from SinGAN were resized to 500 × 500 and then given a predicted segment of the prostate using a pre-trained segmentation neural network. Synthetic images that had a segmentation of the contour that passed our defined criteria were pooled and randomly selected for our human visual assessment.

Figure 5

Model framework diagram of the corresponding generators and discriminators in the SinGAN network.