Automatic spatial estimation of white matter hyperintensities evolution in brain MRI using disease evolution predictor deep neural networks

Abstract

Previous studies have indicated that white matter hyperintensities (WMH), the main radiological feature of small vessel disease, may evolve (i.e., shrink, grow) or stay stable over a period of time. Predicting these changes are challenging because it involves some unknown clinical risk factors that leads to a non-deterministic prediction task. In this study, we propose a deep learning model to predict the evolution of WMH from baseline to follow-up (i.e., 1-year later), namely "Disease Evolution Predictor" (DEP) model, which can be adjusted to become a non-deterministic model. The DEP model receives a baseline image as input and produces a map called "Disease Evolution Map" (DEM), which represents the evolution of WMH from baseline to follow-up. Two DEP models are proposed, namely DEP-UResNet and DEP-GAN, which are representatives of the supervised (i.e., need expert-generated manual labels to generate the output) and unsupervised (i.e., do not require manual labels produced by experts) deep learning algorithms respectively. To simulate the non-deterministic and unknown parameters involved in WMH evolution, we modulate a Gaussian noise array to the DEP model as auxiliary input. This forces the DEP model to imitate a wider spectrum of alternatives in the prediction results. The alternatives of using other types of auxiliary input instead, such as baseline WMH and stroke lesion loads are also proposed and tested. Based on our experiments, the fully supervised machine learning scheme DEP-UResNet regularly performed better than the DEP-GAN which works in principle without using any expert-generated label (i.e., unsupervised). However, a semi-supervised DEP-GAN model, which uses probability maps produced by a supervised segmentation method in the learning process, yielded similar performances to the DEP-UResNet and performed best in the clinical evaluation. Furthermore, an ablation study showed that an auxiliary input, especially the Gaussian noise, improved the performance of DEP models compared to DEP models that lacked the auxiliary input regardless of the model's architecture. To the best of our knowledge, this is the first extensive study on modelling WMH evolution using deep learning algorithms, which deals with the non-deterministic nature of WMH evolution.

Keywords: DEP Generative adversarial network (DEP-GAN); DEP U-Residual Network (DEP-UResNet); Disease evolution predictor (DEP) models; Small vessel disease (SVD); WMH Evolution; White matter hyperintensities (WMH).

Graphical abstract

Fig. 1

“Disease evolution map” (DEM) (right) is produced by subtracting baseline images (middle) from follow-up image (left). In DEM produced by irregularity map (IM) (first row) and probability map (PM) (second row), bright yellow pixels represent positive values (i.e., progression) while dark blue pixels represent negative values (i.e., regression). On the other hand, DEM produced by binary WMH label (LBL) (third row) has three foreground labels which represent progression or “Grow” (green), regression or “Shrink” (red), and “Stable” (blue). We named this special DEM as three-class DEM label (LBL-DEM). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

Schematic of the proposed DEP-GAN with 2 discriminators (critics). M(x) is a generator which generates “fake” disease evolution map (DEM) while C(x) and D(x) are critics to enforce anatomically realistic modifications to the follow-up images and encode realistically plausible DEMs. The flows of “fake” images are shown by the dashed lines. DEP-GAN can take either irregularity map (IM) or probability map (PM) as input. DEP-GAN also has an auxiliary input to deal with the non-deterministic factors in WMH evolution (see Section 2.3 for full explanation).

Fig. 3

Architecture of DEP-GAN, which consists of one generator (upper side, “A”) and two critics (lower side, “C” and “D”). Note how the proposed auxiliary input is feed-forwarded to convolutional layers (yellow, “B”) and then modulated to the generator using FiLM layer (green) inside residual block (ResBlock) (light blue, “E”). Please see Fig. 2 for connections between each part and Section 2.3 for full explanation about auxiliary input. On the other hand, DEP-UResNet is based on DEP-GAN’s generator, including its auxiliary input, with modification of the last non-linear activation function (i.e., use softmax for segmentation instead of tanh). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Examples of real DEM and generated DEMs produced by different GAN architectures for DEP model. From left to right: real DEM and generated DEMs produced by WGAN-GP, VA-GAN, DEP-GAN with 1 critic (DEP-GAN-1C), and DEP-GAN with 2 critics (DEP-GAN-2C) respectively.

Fig. 5

Volumetric agreement (in ml) and correlation (in ICV %) analyses between ground truth (GT) and predicted volume of WMH (Pred) produced by WGAN-GP, VA-GAN, DEP-GAN-1C, and DEP-GAN-2C using (a) IM and (b) PM using Bland-Altman and correlation plots.

Fig. 6

Distributions of DSC scores from all evaluated DEP models in auxiliary input ablation study. These distributions correspond to the Table 4, columns 8–13.

Fig. 7

Qualitative (visual) assessment of DEM produced by DEP-GAN-2C using irregularity map (IM) and DEP-GAN-2C using probability map (PM) with different types/modalities of auxiliary input. The corresponding T2-FLAIR (input data) can be seen in Fig. 9.

Fig. 8

Qualitative (visual) assessment of DEM label produced by DEP-UResNet with different types/modalities of auxiliary input. The corresponding T2-FLAIR (input data) can be seen in Fig. 9.

Fig. 9

Qualitative (visual) assessment of DEM and its corresponding DEM label produced by DEP-GAN-2C using irregularity map (IM) and DEP-GAN-2c using probability map (PM) respectively, with different types/modalities of auxiliary input. The corresponding golden standard of DEM label can be seen in Fig. 8.

Fig. A1

Volumetric agreement analysis (in ml) between ground truth (GT) and predicted volume of WMH with different types/modalities of auxiliary input (Pred) using Bland-Altman plot which correspond to data presented in Table 4. Solid lines correspond to “Vol. Bias” while dashed lines correspond to either “Lower LoA” or “Upper LoA” of the same table. “LoA” stands for limit of agreement.

Fig. A2

Correlation plots between manual WMH volume produced by the expert (GT) and predicted WMH volume by various DEP models with different types/modalities of auxiliary input (Pred). WMH volume is in the percentage of intracranial volume (ICV) to remove any potential bias associated with head size.