MADGAN: unsupervised medical anomaly detection GAN using multiple adjacent brain MRI slice reconstruction

Abstract

Background: Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with either disease stages, various (i.e., more than two types of) diseases, or multi-sequence magnetic resonance imaging (MRI) scans.

Results: We propose unsupervised medical anomaly detection generative adversarial network (MADGAN), a novel two-step method using GAN-based multiple adjacent brain MRI slice reconstruction to detect brain anomalies at different stages on multi-sequence structural MRI: (Reconstruction) Wasserstein loss with Gradient Penalty + 100 [Formula: see text] loss-trained on 3 healthy brain axial MRI slices to reconstruct the next 3 ones-reconstructs unseen healthy/abnormal scans; (Diagnosis) Average [Formula: see text] loss per scan discriminates them, comparing the ground truth/reconstructed slices. For training, we use two different datasets composed of 1133 healthy T1-weighted (T1) and 135 healthy contrast-enhanced T1 (T1c) brain MRI scans for detecting AD and brain metastases/various diseases, respectively. Our self-attention MADGAN can detect AD on T1 scans at a very early stage, mild cognitive impairment (MCI), with area under the curve (AUC) 0.727, and AD at a late stage with AUC 0.894, while detecting brain metastases on T1c scans with AUC 0.921.

Conclusions: Similar to physicians' way of performing a diagnosis, using massive healthy training data, our first multiple MRI slice reconstruction approach, MADGAN, can reliably predict the next 3 slices from the previous 3 ones only for unseen healthy images. As the first unsupervised various disease diagnosis, MADGAN can reliably detect the accumulation of subtle anatomical anomalies and hyper-intense enhancing lesions, such as (especially late-stage) AD and brain metastases on multi-sequence MRI scans.

Keywords: Brain MRI reconstruction; Generative adversarial networks; Self-attention; Unsupervised anomaly detection; Various disease diagnosis.

Fig. 1

Unsupervised medical anomaly detection framework: we train WGAN-GP w/${\ell }_1$ loss on 3 healthy brain axial MRI slices to reconstruct the next 3 ones, and test it on both unseen healthy and abnormal scans to classify them according to average ${\ell }_2$ loss per scan

Fig. 2

Proposed MADGAN architecture for the next 3-slice generation from the input 3 $256\times 176$ brain MRI slices: 3-SA MADGAN has only 3 (red-contoured) SA modules after convolution/deconvolution whereas 7-SA MADGAN has 7 (red- and blue-contoured) SA modules. Similar to RGB images, we concatenate adjacent 3 gray slices into 3 channels

Fig. 3

Example T1 brain MRI slices with CDR = 0/0.5/1/2 from a test set: a Input 3 real slices; b Ground truth next 3 real slices; c, d Next 3 slices reconstructed by MADGAN and 7-SA MADGAN. To compare the real/reconstructed next 3 slices, we show pixelwise ${\ell }_2$ loss values in (b) versus (c) and (b) versus (d) columns, respectively. Using a Jet colormap in [0, 0.2] with alpha-blending, we overlay the obtained maps onto the ground truth slices. The achieved slice-level, pixelwise ${\ell }_2$ loss values are also displayed

Fig. 4

Example T1c brain MRI slices with no abnormal findings/three brain metastases from a test set: a Input 3 real slices; b Ground truth next 3 real slices; c, d Next 3 slices reconstructed by MADGAN and 7-SA MADGAN. To compare the real/reconstructed next 3 slices, we show pixelwise ${\ell }_2$ loss values in (b) versus (c) and (b) versus (d) columns, respectively. Using a Jet colormap in [0, 0.06] with alpha-blending, we overlay the obtained maps onto the ground truth slices. The achieved slice-level, pixelwise ${\ell }_2$ loss values are also displayed

Fig. 5

Example T1c brain MRI slices with four different brain diseases from a test set: a Input 3 real slices; b Ground truth next 3 real slices; c, d Next 3 slices reconstructed by MADGAN and 7-SA MADGAN. To compare the real/reconstructed next 3 slices, we show pixelwise ${\ell }_2$ loss values in (b) versus (c) and (b) versus (d) columns, respectively. Using a Jet colormap in [0, 0.06] with alpha-blending, we overlay the obtained maps onto the ground truth slices. The achieved slice-level, pixelwise ${\ell }_2$ loss values are also displayed

Fig. 6

Distributions of average ${\ell }_2$ loss per scan evaluated on T1 slices with CDR = 0/0.5/1/2 reconstructed by: a MADGAN and b 7-SA MADGAN

Fig. 7

Distributions of average ${\ell }_2$ loss per scan evaluated on T1c slices with no abnormal findings/brain metastases/various diseases reconstructed by: a MADGAN and b 7-SA MADGAN

Fig. 8

AUC performance on T1 scans using average ${\ell }_2$ loss per scan under different training steps (i.e., 150k, 300k, 600k, 900k, 1.8M steps). Unchanged CDR = 0 (i.e., cognitively healthy population) is compared against: a all the other CDRs (i.e., dementia); b CDR = 0.5 (i.e., very mild dementia); c CDR = 1 (i.e., mild dementia); d CDR = 2 (i.e., moderate dementia)

Fig. 9

AUC performance on T1c scans using average ${\ell }_2$ loss per scan under different training steps (i.e., 150k, 300k, 600k, 900k, 1.8M steps). No abnormal findings are compared against: a brain metastases + various diseases; b brain metastases; c various diseases

Fig. 10

Unsupervised anomaly detection results using average ${\ell }_2$ loss per scan on reconstructed T1 slices (ROCs and AUCs). Unchanged CDR = 0 (i.e., cognitively healthy population) is compared against: a all the other CDRs (i.e., dementia); b CDR = 0.5 (i.e., very mild dementia); c CDR = 1 (i.e., mild dementia); d CDR = 2 (i.e., moderate dementia). Each model is trained for 1.8M steps

Fig. 11

Unsupervised anomaly detection results using average ${\ell }_2$ loss per scan on reconstructed T1c slices (ROCs and AUCs). No abnormal findings are compared against: a brain metastases + various diseases; b brain metastases; c various diseases. Each model is trained for 1.8M steps