Balancing Data Quality and Bias: Investigating Functional Connectivity Exclusions in the Adolescent Brain Cognitive Development℠ (ABCD Study) Across Quality Control Pathways

Abstract

Analysis of resting state fMRI (rs-fMRI) typically excludes images substantially degraded by subject motion. However, data quality, including degree of motion, relates to a broad set of participant characteristics, particularly in pediatric neuroimaging. Consequently, when planning quality control (QC) procedures researchers must balance data quality concerns against the possibility of biasing results by eliminating data. In order to explore how researcher QC decisions might bias rs-fMRI findings and inform future research design, we investigated how a broad spectrum of participant characteristics in the Adolescent Brain and Cognitive Development (ABCD) study were related to participant inclusion/exclusion across versions of the dataset (the ABCD Community Collection and ABCD Release 4) and QC choices (specifically, motion scrubbing thresholds). Across all these conditions, we found that the odds of a participant's exclusion related to a broad spectrum of behavioral, demographic, and health-related variables, with the consequence that rs-fMRI analyses using these variables are likely to produce biased results. Consequently, we recommend that missing data be formally accounted for when analyzing rs-fMRI data and interpreting results. Our findings demonstrate the urgent need for better data acquisition and analysis techniques which minimize the impact of motion on data quality. Additionally, we strongly recommend including detailed information about quality control in open datasets such as ABCD.

Keywords: ABCD; adolescents; missing data; motion; quality control; rs‐fMRI.

FIGURE 1

9 samples were generated based on quality control conditions. (A) shows the path diagram leading to the various conditions. (B) Shows n by Condition. (C) Illustrates non‐overlap between the ABCD Tabulated, ABCD Recommended, and ABCC conditions.

FIGURE 2

Categorical variables by inclusion criteria.

FIGURE 3

Continuous variables (standardized) by condition.

FIGURE 4

Some sites showed higher rates of exclusion in ABCC conditions versus ABCD Recommended, and some of those sites (e.g., LA, Baltimore, San Diego) had higher numbers of non‐white participants.

FIGURE 5

The proportion of cases missing in excess of the sample average (y axis) against the percentage of data missing from the sample (with conditions labeled—x axis). The steepest gains in bias occur early in thresholding—as more data is excluded, biases decelerate and then self‐correct below around 0.1 mm thresholding. Additional variables are plotted in the Supporting Information.

FIGURE 6

Hexagonal binned density plots of the correlation between participants' functional connectivity in each pair of regions in the HCP 2016 cortical atlas and FD (QC‐FC), plotted against the average Euclidean distance between said regions. Data from all ABCC participants is included (n = 9600).