Leveraging the adolescent brain cognitive development study to improve behavioral prediction from neuroimaging in smaller replication samples

Abstract

Neuroimaging is a popular method to map brain structural and functional patterns to complex human traits. Recently published observations cast doubt upon these prospects, particularly for prediction of cognitive traits from structural and resting state functional magnetic resonance imaging (MRI). We leverage baseline data from thousands of children in the Adolescent Brain Cognitive DevelopmentSM Study to inform the replication sample size required with univariate and multivariate methods across different imaging modalities to detect reproducible brain-behavior associations. We demonstrate that by applying multivariate methods to high-dimensional brain imaging data, we can capture lower dimensional patterns of structural and functional brain architecture that correlate robustly with cognitive phenotypes and are reproducible with only 41 individuals in the replication sample for working memory-related functional MRI, and ~ 100 subjects for structural and resting state MRI. Even with 100 random re-samplings of 100 subjects in discovery, prediction can be adequately powered with 66 subjects in replication for multivariate prediction of cognition with working memory task functional MRI. These results point to an important role for neuroimaging in translational neurodevelopmental research and showcase how findings in large samples can inform reproducible brain-behavior associations in small sample sizes that are at the heart of many research programs and grants.

Keywords: brain-behavior associations; multivariate modeling; neurocognition; structural MRI; task functional MRI.

Fig. 1

Univariate associations, estimated with Pearson r correlations, between general cognition and: A. Five vertex-wise cortical features; and B. Resting state correlation data across 333 cortical regions from the Gordon parcellation. In panel B, regions are clustered by resting-state network for visualization purposes only. Abbreviations: SA, surface area; CT, cortical thickness; RND, restricted directional diffusion within superficial white matter; RNI, restricted isotropic diffusion intracortically; ENb, emotional N-back task fMRI, reflecting the 2- vs. 0-back contrast. RS, resting state fMRI. Aud, Auditory; CingOp. Cingulo-Opercular; CingPar. Cingulo-parietal; Def. Default mode; DorAtt. Dorsal attention; FrPar,fronto-parietal; RsTemp. Retrosplenial temporal; Sal. Salience; SMhand. Sensorimotor hand; SMmouth. Sensorimotor mouth; VenAtt, ventral attention; Vis. Visual.

Fig. 2

Comparison of out-of-sample prediction performance from multivariate analyses (panel A) per imaging measure/modality, vs. maximum absolute correlations derived from univariate analyses (panel B). Error bars reflect standard deviation, adjusted for the 10% sample overlap in test datasets. Numbers above each bar reflect the sample size required to achieve 80% power to detect effects in a replication sample, given the uncovered r values from the ABCD discovery sample. Abbreviations: sMRI, structural MRI; dMRI, diffusion MRI; fMRI, functional MRI; SA, surface area; CT, cortical thickness; RND, restricted directional diffusion within superficial white matter; RNI, restricted isotropic diffusion intracortically; ENb, emotional N-back task fMRI, reflecting the 2- vs. 0-back contrast; RS, resting state fMRI.

Fig. 3

Power curves displaying replication sample sizes required (x-axis: Log-scale N) to achieve desired level of power (y-axis) based on performance of each imaging measure in predicting general cognition in replication sample using (A) multivariate vs (B) univariate methods. Abbreviations: SA, surface area; CT, cortical thickness; RND, restricted directional diffusion within superficial white matter; RNI, restricted isotropic diffusion intracortically; ENb, emotional N-back task fMRI, reflecting the 2- vs. 0-back contrast; RS, resting state fMRI.

Fig. 4

Replication curves showing out-of-sample prediction performance (defined by r on the y-axis) for each of the six imaging measures predicting general cognition, as a function of sample size in the discovery sample (represented on the x-axis in log-scale units). Replication sample size was fixed to be n = 1,000 across discovery sample sizes. For prediction with the fMRI-derived features, there were not enough participants left over at a discovery sample size of n = 5,000 to reach a replication sample of n = 1,000. Thus we only estimated out-of-sample performance up to a discovery sample size of n = 2,299 for these features. Multivariate metrics are compared to univariate r-values, reflecting the absolute maximum correlation value. Error bars reflect standard deviation, adjusted for sample overlap in the replication datasets, and are jittered for better visualization. Abbreviations: sMRI. Structural MRI; dMRI. Diffusion MRI; fMRI. Functional MRI; SA. Surface area; CT. Cortical thickness; RND. restricted directional diffusion within superficial white matter; RNI. Restricted isotropic diffusion intracortically; ENb. Emotional N-back task fMRI, reflecting the 2- vs. 0-back contrast; RS. Resting state fMRI.