Scan-rescan reliability of subcortical brain volumes derived from automated segmentation

Abstract

Large-scale longitudinal studies of regional brain volume require reliable quantification using automated segmentation and labeling. However, repeated MR scanning of the same subject, even if using the same scanner and acquisition parameters, does not result in identical images due to small changes in image orientation, changes in prescan parameters, and magnetic field instability. These differences may lead to appreciable changes in estimates of volume for different structures. This study examined scan-rescan reliability of automated segmentation algorithms for measuring several subcortical regions, using both within-day and across-day comparison sessions in a group of 23 normal participants. We found that the reliability of volume measures including percent volume difference, percent volume overlap (Dice's coefficient), and intraclass correlation coefficient (ICC), varied substantially across brain regions. Low reliability was observed in some structures such as the amygdala (ICC = 0.6), with higher reliability (ICC = 0.9) for other structures such as the thalamus and caudate. Patterns of reliability across regions were similar for automated segmentation with FSL/FIRST and FreeSurfer (longitudinal stream). Reliability was associated with the volume of the structure, the ratio of volume to surface area for the structure, the magnitude of the interscan interval, and the method of segmentation. Sample size estimates for detecting changes in brain volume for a range of likely effect sizes also differed by region. Thus, longitudinal research requires a careful analysis of sample size and choice of segmentation method combined with a consideration of the brain structure(s) of interest and the magnitude of the anticipated effects.

Figure 1

Scatter plots showing correlation between segmented amygdala volumes (mm³) from scan 1A and 1B for FSL/FIRST and FreeSurfer. Left hemisphere volumes are in green and right hemisphere volumes in orange. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 2

Scatter plots showing correlation between segmented hippocampus volumes (mm³) from scan 1A and 1B for FSL/FIRST and FreeSurfer. Left hemisphere volumes are in green and right hemisphere volumes in orange. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 3

Percent volume difference for scans with a 1‐h and 1‐week interscan interval for nine subcortical brain structures segmented with FIRST and FreeSurfer. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 4

Percent volume overlap for scans with a 1‐h and 1‐week interscan interval for nine subcortical brain structures segmented with FIRST and FreeSurfer. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 5

The T1 images and segmentations for subject 7 are shown for scan 1A (left panel) and for scan 1B (center panel). Segmentation of the L‐putamen (circled) is dramatically different on the lateral surface for scan 1A (3,777 voxels) compared to scan 1B (7,317 voxels). This resulted in lower intraclass correlations for 1A1B (0.29) and 1A2A (0.29) as compared to 2A2B (0.96) and 1B2B (0.95). There is no obvious artifact visible in scan 1A that might explain this discrepancy. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 6

Sample size requirements (y‐axis) for FreeSurfer (left) and FSL/FIRST (right) assuming a within subject design with two observations to achieve 80% power and 5% alpha level are shown for a range of effect sizes (x‐axis) and each of the nine subcortical structures. Note that the sample size is scaled by log₁₀ to enhance visualization of curves at higher effect sizes. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]