The impact of quality control on cortical morphometry comparisons in autism

Abstract

Structural magnetic resonance imaging (MRI) quality is known to impact and bias neuroanatomical estimates and downstream analysis, including case-control comparisons, and a growing body of work has demonstrated the importance of careful quality control (QC) and evaluated the impact of image and image-processing quality. However, the growing size of typical neuroimaging datasets presents an additional challenge to QC, which is typically extremely time and labour intensive. One of the most important aspects of MRI quality is the accuracy of processed outputs, which have been shown to impact estimated neurodevelopmental trajectories. Here, we evaluate whether the quality of surface reconstructions by FreeSurfer (one of the most widely used MRI processing pipelines) interacts with clinical and demographic factors. We present a tool, FSQC, that enables quick and efficient yet thorough assessment of outputs of the FreeSurfer processing pipeline. We validate our method against other existing QC metrics, including the automated FreeSurfer Euler number, two other manual ratings of raw image quality, and two popular automated QC methods. We show strikingly similar spatial patterns in the relationship between each QC measure and cortical thickness; relationships for cortical volume and surface area are largely consistent across metrics, though with some notable differences. We next demonstrate that thresholding by QC score attenuates but does not eliminate the impact of quality on cortical estimates. Finally, we explore different ways of controlling for quality when examining differences between autistic individuals and neurotypical controls in the Autism Brain Imaging Data Exchange (ABIDE) dataset, demonstrating that inadequate control for quality can alter results of case-control comparisons.

Keywords: FreeSurfer; autism; cortical thickness; quality control; structural MRI.

Fig. 1.

FSQC image generation workflow. From left to right: T1 images were processed with FreeSurfer 6.0.1 and displayed in FreeView with pial and white matter surfaces overlaid on the T1 image (both hemispheres). Screenshots were automated and taken at predefined, consistent coordinates, for a total of 10 images per participant. Images were then displayed and rated in the Image-Rating app, and scores were averaged across all 10 images for each participant.

Fig. 2.

(A) Inter-rater correlation matrix for FSQC ratings for a subset of 50 participants (500 images). All pairs of raters were significantly correlated with each other between 0.7-0.8 rho. (B) Correlations between different QC metrics. Because Qoala-T is reverse coded relative to the other metrics, the absolute values are shown for the Qoala-T correlations. All measures were significantly correlated with each other. (C) Relationship between FSQC and age. A significant effect of age was observed in which younger participants had lower quality ratings. (D) FSQC score distributions by site. There was significant variability in quality across sites. (E) Box and violin plot of FSQC distributions for males and females. There was no significant sex difference in FSQC. (F) Box and violin plot of FSQC distributions by diagnosis. Autistic participants had significantly higher FSQC scores (i.e., lower image quality) relative to controls (p < 0.0001, d = -0.27). Box plots represent the interquartile range, the middle line denotes the median, and the black dot represents the mean. The curves of the violin plots show the distribution and density estimate of FSQC scores for each group.

Fig. 3.

Associations between QC metrics and regional cortical morphometry. There was a significant relationship between image quality and neuroanatomical estimates across much of the cortex for all metrics and phenotypes. Relationships were largely negative and strongest for cortical thickness. Spatial patterning of results was highly similar across most metrics, with the exception of SA and CV for Qoala-T, which showed largely positive relationships, in contrast to the other metrics.

Fig. 4.

Relationship between cortical thickness and FSQC (left) and Euler number (right) after thresholding at different levels of stringency. Accompanying graphs show the attenuation of both number of significant regions observed (top) and partial correlation effect size (bottom two panels) as stringency increases.

Fig. 5.

Impact of autism diagnosis on cortical thickness (Cohen’s d) without accounting for image quality (A), when controlling for FSQC (B) or Euler (C), and thresholding by FSQC (D) and Euler (E). Significant regions passing 5% FDR are shown with a black border; other regions are subthreshold (i.e., not surviving FDR) differences. Most results indicate thicker cortex in autism relative to controls; results do not change drastically with quality control, but most negative associations between diagnosis and CT (autism < controls) disappear. Significantly thicker cortex in the superior temporal gyrus, which has previously been reported in autism, is observed only when controlling for quality (FSQC or Euler).