Addressing the Generalizability of AI in Radiology Using a Novel Data Augmentation Framework with Synthetic Patient Image Data: Proof-of-Concept and External Validation for Classification Tasks in Multiple Sclerosis

Abstract

Artificial intelligence (AI) models often face performance drops after deployment to external datasets. This study evaluated the potential of a novel data augmentation framework based on generative adversarial networks (GANs) that creates synthetic patient image data for model training to improve model generalizability. Model development and external testing were performed for a given classification task, namely the detection of new fluid-attenuated inversion recovery lesions at MRI during longitudinal follow-up of patients with multiple sclerosis (MS). An internal dataset of 669 patients with MS (n = 3083 examinations) was used to develop an attention-based network, trained both with and without the inclusion of the GAN-based synthetic data augmentation framework. External testing was performed on 134 patients with MS from a different institution, with MR images acquired using different scanners and protocols than images used during training. Models trained using synthetic data augmentation showed a significant performance improvement when applied on external data (area under the receiver operating characteristic curve [AUC], 83.6% without synthetic data vs 93.3% with synthetic data augmentation; P = .03), achieving comparable results to the internal test set (AUC, 95.0%; P = .53), whereas models without synthetic data augmentation demonstrated a performance drop upon external testing (AUC, 93.8% on internal dataset vs 83.6% on external data; P = .03). Data augmentation with synthetic patient data substantially improved performance of AI models on unseen MRI data and may be extended to other clinical conditions or tasks to mitigate domain shift, limit class imbalance, and enhance the robustness of AI applications in medical imaging. Keywords: Brain, Brain Stem, Multiple Sclerosis, Synthetic Data Augmentation, Generative Adversarial Network Supplemental material is available for this article. © RSNA, 2024.

Keywords: Brain; Brain Stem; Generative Adversarial Network; Multiple Sclerosis; Synthetic Data Augmentation.

Figure 1:

(A) Schematic representation of the process for generation of synthetic MS patient data. Briefly, T1 data from the OASIS dataset was coregistered to a matched template patient T1 data from the same training fold. The registered T1 data were then fed to the FAST segmentation algorithm to generate three-dimensional segmentations of the anatomic compartments (ie, white matter, gray matter, cerebrospinal fluid). Simultaneously, the corresponding FLAIR image from the matched patient was fed to a U-Net lesion segmentation algorithm to produce MS lesion segmentation maps. Two different versions of the lesion maps (with and without random elimination of lesion volumes) were generated and merged with the brain segmentation. The registered T1 and the merged segmentation maps are then fed to the cGAN model to create synthetic FLAIR images based on the patient’s template. (B) Three example output cases of synthetic patient data. Yellow arrows indicate simulated new lesions in the synthetic follow-up examination. cGAN = conditional generative adversarial network, FLAIR = fluid-attenuated inversion recovery, MS = multiple sclerosis.

Figure 2:

ROC curves with performance of the developed models across all datasets. Our attention-based model achieved the significantly higher performance as compared with the simpler ResNet architecture (model 1, P < .001). Performance of the attention-based model significantly improved on the external UKB test set after the inclusion of synthetic data augmentation, demonstrating similar performance compared with the internal test set (P = .53). AUC = area under the ROC curve, CNN = convolutional neural network, ROC = receiver operating characteristic, UKB = Bonn University Hospital.