Synthesizing images of tau pathology from cross-modal neuroimaging using deep learning

Abstract

Given the prevalence of dementia and the development of pathology-specific disease-modifying therapies, high-value biomarker strategies to inform medical decision-making are critical. In vivo tau-PET is an ideal target as a biomarker for Alzheimer's disease diagnosis and treatment outcome measure. However, tau-PET is not currently widely accessible to patients compared to other neuroimaging methods. In this study, we present a convolutional neural network (CNN) model that imputes tau-PET images from more widely available cross-modality imaging inputs. Participants (n = 1192) with brain T1-weighted MRI (T1w), fluorodeoxyglucose (FDG)-PET, amyloid-PET and tau-PET were included. We found that a CNN model can impute tau-PET images with high accuracy, the highest being for the FDG-based model followed by amyloid-PET and T1w. In testing implications of artificial intelligence-imputed tau-PET, only the FDG-based model showed a significant improvement of performance in classifying tau positivity and diagnostic groups compared to the original input data, suggesting that application of the model could enhance the utility of the metabolic images. The interpretability experiment revealed that the FDG- and T1w-based models utilized the non-local input from physically remote regions of interest to estimate the tau-PET, but this was not the case for the Pittsburgh compound B-based model. This implies that the model can learn the distinct biological relationship between FDG-PET, T1w and tau-PET from the relationship between amyloid-PET and tau-PET. Our study suggests that extending neuroimaging's use with artificial intelligence to predict protein specific pathologies has great potential to inform emerging care models.

Keywords: Alzheimer’s disease; FDG PET; cross-modality imputation; deep learning; tau PET.

Figure 1

Dense-U-Net architecture and layout of analysis. (A) The architecture receives input of size 128 × 128 × 128 and produces the artificial intelligence (AI)-imputed tau-PET of the same dimension with input data. Dense-U-net architecture is composed of encoder (left), decoder (right) and bridge. Left dotted box illustrates a layout of dense connection in dense block, when output from each rectified linear unit (ReLU) layer is concatenated (circular C) to the input of the block before fed to the next layer. The numbers denoted above the dense blocks indicate a number of filters. (B) The similarity between ground-truth tau PET and AI-imputed tau PET was assessed across total participants in test set using a regional standardized uptake value ratio (SUVR) calculated from 46 regions of interest (ROIs) and meta-ROI. Pearson’s correlation and mean absolute percentage error (MAPE) were used as an evaluation metric.

Figure 2

FDG-PET based tau-PET synthesis results. (A) Eight representative cases with original FDG-PET, ground-truth tau-PET and artificial intelligence (AI)-imputed tau PET. (B) Scatter plots of ground-truth tau-PET and AI-imputed tau PET from seven representative regions of interest (ROIs) and meta-ROI. r indicates Pearson’s correlation coefficient. MAPE = mean absolute percentage error. Linear regression (black line) and 95% confidence bands (dotted lines) are shown. (C) The mean of correlation coefficient and MAPE of five folds from 46 ROIs and the meta-ROI is summarized in a box plot. The yellow-coloured box depicts the meta-ROI result. Open circles indicate different folds. SUVR = standardized uptake value ratio.

Figure 3

Structural MRI-based tau-PET synthesis results. (A) Eight representative cases with original T₁-weighted (T1w), ground-truth tau-PET and artificial intelligence (AI)-imputed tau-PET. (B) Scatter plots between ground-truth tau-PET and AI-imputed tau-PET from seven representative regions of interest (ROIs) and meta-ROI. r indicates the Pearson’s correlation coefficient. MAPE = mean absolute percentage error. Linear regression (black line) and 95% confidence bands (dotted lines) are shown. (C) The mean correlation coefficient and MAPE of five folds from 46 ROIs and meta-ROI is summarized in the box plots. The yellow-coloured box depicts the meta-ROI result. Open circles indicate different folds. SUVR = standardized uptake value ratio.

Figure 4

Amyloid-PET based tau-PET synthesis results. (A) Eight representative cases with actual Pittsburgh compound B (PiB)-PET, ground-truth tau-PET and artificial intelligence (AI)-imputed tau PET. (B) Scatter plots between ground-truth tau PET and AI-imputed tau PET from seven representative regions of interest (ROIs) and meta-ROI. r indicates the Pearson’s correlation coefficient. MAPE = mean absolute percentage error. Linear regression (black line) and 95% confidence bands (dotted lines) are shown. (C) The mean of correlation coefficient and MAPE of five folds from 46 ROIs and meta-ROI is summarized in a box plot. The yellow-coloured box depicts the meta-ROI result. Open circles indicate different folds. SUVR = standardized uptake value ratio.

Figure 5

ROC analysis for tau PET positivity. Tau positivity predicted from the ground-truth tau-PET using four different meta-ROI (region of interest) cut-off thresholds (1.11, 1.21, 1.33 and 1.46) were obtained using six different predictors: (A) actual FDG-PET and (B) FDG-based synthesized tau-PET with (C) area under the ROC curve (AUROC) comparison between the original FDG and FDG-based AI-imputed tau-PET; (D) cortical thickness from the cohort who had GE scans and (E) T₁-weighted (T1W)-based artificial intelligence (AI)-imputed tau-PET from the cohort who had GE scans with (F) AUROC comparison between the cortical thickness and T1W-based AI-imputed tau-PET. (G) Actual PiB-PET and (H) PiB-based AI-imputed tau-PET with (I) AUROC comparison between the PiB-PET and PiB-based AI-imputed tau-PET. A pair-wise comparison was performed between input data and the corresponding AI-imputed tau PET for each cut-off. Statistical significance was tested by post hoc Holm-Sidak comparisons after two-way ANOVA. **P < 0.005, ****P < 0.0001. Open circles in C, F and I indicate different folds. ns = not significant; ROC = receiver operating characteristic.

Figure 6

Diagnostic performance of AI-imputed tau-PET. (A–D) Meta-ROI (region of interest) standardized uptake value ratios (SUVRs) from the actual tau, FDG-, T₁-weighted (T1W)-, and Pittsburgh compound B (PiB)-based artificial intelligence (AI)-imputed tau-PET were plotted for each diagnostic group. Red, blue and grey coloured dots show amyloid positive, negative and unknown, respectively. (E–G) Receiver operating characteristic (ROC) analysis was performed for classifying the diagnostic groups using seven predictors. Open circles indicate different folds. Statistical significance was assessed with two-way ANOVA and Holm-Sidak post hoc comparison. Aβ = amyloid-β; CUA = cognitively unimpaired with normal amyloid; CUA+ = cognitively unimpaired with abnormal amyloid level; MCI = mild cognitively impaired; AD = Alzheimer’s disease; LPA = logopenic progressive aphasia; PCA = posterior cortical atrophy; PSP = progressive supranuclear palsy; FTD = frontotemporal dementia; SD = semantic dementia; nfvPAA = non-fluent variant of progressive associative agnosia; RBD = REM sleep behaviour disorder; DLB = dementia with Lewy bodies; spec = spectrum. **P < 0.01, ***P < 0.0001.

Figure 7

Occlusion analysis. Region of interest (ROI)-wise occlusion analysis was performed to enhance the interpretability of model. (A, C and E) The adjacency matrix shows the ΔMAPE (mean absolute percentage error) in one ROI (horizontal axis) from occluding another ROI (vertical axis) for FDG-, T1W- and PiB-based model, respectively. ΔMAPE was calculated as MAPE_R1→R2 − MAPE_R2, where R1 is an occluded ROI and R2 is the region where the MAPE is calculated. The right panel in each matrix indicates the summation of ΔMAPE along the horizontal axis. (B, D and F) 3D rendering plots of the adjacency matrix in A, C and E for FDG-, T1W- and PiB-based model, respectively. Each edge’s colour was illustrated by ΔMAPE value between nodes. Each label denoted above the figure indicate the occluded regions. BG&Thal = basal ganglia and thalamus; CG = cingulate cortex; FL = frontal lobe; MTL = medial temporal lobe; OL = occipital lobe; PL = parietal lobe; SMC = sensorimotor cortex; TL = temporal lobe.