Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun 15;44(9):3481-3492.
doi: 10.1002/hbm.26292. Epub 2023 Apr 5.

Probing multiple algorithms to calculate brain age: Examining reliability, relations with demographics, and predictive power

Eva Bacas  1 , Isabella Kahhalé  1   2 , Pradeep R Raamana  3 , Julian B Pablo  1 , Apurvaa S Anand  1 , Jamie L Hanson  1   2
Affiliations

Probing multiple algorithms to calculate brain age: Examining reliability, relations with demographics, and predictive power

Eva Bacas et al. Hum Brain Mapp. .

Abstract

The calculation of so-called "brain age" from structural MRIs has been an emerging biomarker in aging research. Data suggests that discrepancies between chronological age and the predicted age of the brain may be predictive of mortality and morbidity (for review, see Cole, Marioni, Harris, & Deary, 2019). However, with these promising results come technical complexities of how to calculate brain age. Various groups have deployed methods leveraging different statistical approaches, often crafting novel algorithms for assessing this biomarker derived from structural MRIs. There remain many open questions about the reliability, collinearity, and predictive power of different algorithms. Here, we complete a rigorous systematic comparison of three commonly used, previously published brain age algorithms (XGBoost, brainageR, and DeepBrainNet) to serve as a foundation for future applied research. First, using multiple datasets with repeated structural MRI scans, we calculated two metrics of reliability (intraclass correlations and Bland-Altman bias). We then considered correlations between brain age variables, chronological age, biological sex, and image quality. We also calculated the magnitude of collinearity between approaches. Finally, we used machine learning approaches to identify significant predictors across brain age algorithms related to clinical diagnoses of cognitive impairment. Using a large sample (N = 2557), we find all three commonly used brain age algorithms demonstrate excellent reliability (r > .9). We also note that brainageR and DeepBrainNet are reasonably correlated with one another, and that the XGBoost brain age is strongly related to image quality. Finally, and notably, we find that XGBoost brain age calculations were more sensitive to the detection of clinical diagnoses of cognitive impairment. We close this work with recommendations for future research studies focused on brain age.

Keywords: aging; brain age; neuroscience; reliability; statistics.

PubMed Disclaimer

Figures

FIGURE 1
FIGURE 1
Distribution of participant age for the different projects we leveraged in our analyses. The horizontal axis depicts participant age in years, while the vertical axis shows the number of participants within a given age bin. Each dataset is shown in a different color, with AOMIC shown in red, HCP‐A shown in green, and OASIS‐3 shown in blue.
FIGURE 2
FIGURE 2
Density Plot of Differences in 3 Brain Age Algorithms across the AOMIC sample. The horizontal axis shows the portion of differences between repeated scans (as a percentage). The vertical axis is the density (or frequency) of such bias. Each brain age algorithm is shown in a different color with XGBoost shown in light red, brainageR shown in light green, and DeepBrainNet shown in light blue.
FIGURE 3
FIGURE 3
Density Plot of Differences in 3 Brain Age Algorithms across the OASIS sample. The horizontal axis shows the portion of differences between repeated scans (as a percentage). The vertical axis is the density (or frequency) of such bias. Each brain age algorithm is shown in a different color with XGBoost shown in light red, brainageR shown in light green, and DeepBrainNet shown in light blue.
FIGURE 4
FIGURE 4
Scatterplots of the relationship between brain age and real age across all algorithms. There are three panels, each representing a different brain age algorithm—XGBoost is on the far‐left panel, brainageR is in the middle, and DeepBrainNet is on the far‐right panel. The horizontal axis shows participant chronological (real) age, while the vertical axis represents predicted brain age. In each panel, red dots represent female participants, and teal dots represent male participants.
FIGURE 5
FIGURE 5
Correlation plot between brain age, brain age delta, chronological age, and CAT12 score across algorithms. There are rows and columns representing different relevant variables. The correlation between variables is shown at the confluence of a row and a column. The strength of a correlation is represented by the color of the background, and the exact value is written in black text. Negative correlations colored red, with strong negative correlations in dark red and weak negative correlations in light red. Positive correlations are colored blue, with strong positive correlations in dark blue and weak positive correlations in light blue. Weak or no correlation is colored white. BA = brain age.
FIGURE 6
FIGURE 6
Caption: ROC curve for the final model, trained across 10 repeats.
FIGURE 7
FIGURE 7
A confusion matrix for the EN predictions. True negative = 641. False negative = 22. False positive = 42. True positive = 18.
See this image and copyright information in PMC

References

    1. Aubert, G. , & Lansdorp, P. M. (2008). Telomeres and aging. Physiological Reviews, 88(2), 557–579. 10.1152/physrev.00026.2007 - DOI - PubMed
    1. Avesani, P. , McPherson, B. , Hayashi, S. , Caiafa, C. F. , Henschel, R. , Garyfallidis, E. , Kitchell, L. , Bullock, D. , Patterson, A. , & Olivetti, E. (2019). The open diffusion data derivatives, brain data upcycling via integrated publishing of derivatives and reproducible open cloud services. Scientific Data, 6(1), 1–13. 10.1038/s41597-019-0073-y - DOI - PMC - PubMed
    1. Bashyam, V. M. , Shou, H. , & Davatzikos, C. (2021). Reply: From 'loose fitting'to high‐performance, uncertainty‐aware brain‐age modelling. Brain, 144, e32. 10.1093/brain/awaa455 - DOI - PubMed
    1. Beck, D. , de Lange, A.‐M. G. , Maximov, I. I. , Richard, G. , Andreassen, O. A. , Nordvik, J. E. , & Westlye, L. T. (2021). White matter microstructure across the adult lifespan: A mixed longitudinal and cross‐sectional study using advanced diffusion models and brain‐age prediction. Neuroimage, 224, 117441. 10.1016/j.neuroimage.2020.117441 - DOI - PubMed
    1. Bookheimer, S. Y. , Salat, D. H. , Terpstra, M. , Ances, B. M. , Barch, D. M. , Buckner, R. L. , Burgess, G. C. , Curtiss, S. W. , Diaz‐Santos, M. , Elam, J. S. , Fischl, B. , Greve, D. N. , Hagy, H. A. , Harms, M. P. , Hatch, O. M. , Hedden, T. , Hodge, C. , Japardi, K. C. , Kuhn, T. P. , … Yacoub, E. (2019). The lifespan human connectome project in aging: An overview. NeuroImage, 185, 335–348. 10.1016/j.neuroimage.2018.10.009 - DOI - PMC - PubMed

Publication types