Validating Automated Segmentation Tools in the Assessment of Caudate Atrophy in Huntington's Disease

Abstract

Background: Neuroimaging shows considerable promise in generating sensitive and objective outcome measures for therapeutic trials across a range of neurodegenerative conditions. For volumetric measures the current gold standard is manual delineation, which is unfeasible for samples sizes required for large clinical trials. Methods: Using a cohort of early Huntington's disease (HD) patients (n = 46) and controls (n = 35), we compared the performance of four automated segmentation tools (FIRST, FreeSurfer, STEPS, MALP-EM) with manual delineation for generating cross-sectional caudate volume, a region known to be vulnerable in HD. We then examined the effect of each of these baseline regions on the ability to detect change over 15 months using the established longitudinal Caudate Boundary Shift Integral (cBSI) method, an automated longitudinal pipeline requiring a baseline caudate region as an input. Results: All tools, except Freesurfer, generated significantly smaller caudate volumes than the manually derived regions. Jaccard indices showed poorer levels of overlap between each automated segmentation and manual delineation in the HD patients compared with controls. Nevertheless, each method was able to demonstrate significant group differences in volume (p < 0.001). STEPS performed best qualitatively as well as quantitively in the baseline analysis. Caudate atrophy measures generated by the cBSI using automated baseline regions were largely consistent with those derived from a manually segmented baseline, with STEPS providing the most robust cBSI values across both control and HD groups. Conclusions: Atrophy measures from the cBSI were relatively robust to differences in baseline segmentation technique, suggesting that fully automated pipelines could be used to generate outcome measures for clinical trials.

Keywords: FIRST; FreeSurfer; Huntington (disease); MALP-EM; STEPS; automated segmentation; caudate.

Figure 1

Boxplots showing caudate volumes separated by group for each segmentation tool including manual segmentation. Boxes show first quartile, median and third quartile with whiskers representing the smallest and largest volumes. Dots represent outliers. Independent t-tests for each method demonstrated significant volume differences at 95% CI between the two groups, all p < 0.0001.

Figure 2

Boxplot demonstrating caudate volume outputs by disease state (HD=1, Controls=0) and site. Boxes show first quartile, median and third quartile with whiskers represent the smallest and largest volumes. Dots represent outliers. More outliers were found at the Ulm site.

Figure 3

Boxplot demonstrating Jaccard Indices (as a ratio) for each automated method ROI with manual ROI. Boxes show first quartile, median and third quartile with whiskers represent the smallest and largest volumes. Dots represent outliers. STEPS segmented ROIs had larger overlaps with manually segmented ROI in both controls and HD subjects (closest to 1).