Comparison of AdaBoost and support vector machines for detecting Alzheimer's disease through automated hippocampal segmentation

Abstract

We compared four automated methods for hippocampal segmentation using different machine learning algorithms: 1) hierarchical AdaBoost, 2) support vector machines (SVM) with manual feature selection, 3) hierarchical SVM with automated feature selection (Ada-SVM), and 4) a publicly available brain segmentation package (FreeSurfer). We trained our approaches using T1-weighted brain MRIs from 30 subjects [10 normal elderly, 10 mild cognitive impairment (MCI), and 10 Alzheimer's disease (AD)], and tested on an independent set of 40 subjects (20 normal, 20 AD). Manually segmented gold standard hippocampal tracings were available for all subjects (training and testing). We assessed each approach's accuracy relative to manual segmentations, and its power to map AD effects. We then converted the segmentations into parametric surfaces to map disease effects on anatomy. After surface reconstruction, we computed significance maps, and overall corrected p-values, for the 3-D profile of shape differences between AD and normal subjects. Our AdaBoost and Ada-SVM segmentations compared favorably with the manual segmentations and detected disease effects as well as FreeSurfer on the data tested. Cumulative p-value plots, in conjunction with the false discovery rate method, were used to examine the power of each method to detect correlations with diagnosis and cognitive scores. We also evaluated how segmentation accuracy depended on the size of the training set, providing practical information for future users of this technique.

Fig. 1

Discrete AdaBoost algorithm. 1 is an indicator function.

Fig. 2

Procedure for Ada-SVM tree training. The AdaBoost tree is trained in the same way except that it does not use SVM for classification – it uses traditional AdaBoost for classification.

Fig. 3

Example hippocampal segmentations from each of the methods being compared: manual tracing, Ada-SVM, AdaBoost, and FreeSurfer [14]. The left hippocampus is shown in yellow, and the right hippocampus is shown in green. Ada-SVM gives smoother, more spatially coherent result than any of the other methods, and even appears slightly less noisy than the manual traces, which are typically created in coronal sections and may appear jagged when resliced in other planes. All methods give anatomically reasonable segmentations, but some give highly irregular or noisy boundaries. Here we show only the test cases for Ada-SVM and AdaBoost, because we wish to evaluate their performance on unseen images, not on the same manually segmented images that were used for training. The brain MRI quality here is typical of those used in AD morphometric studies, showing widespread atrophy and moderate to poor gray/white matter contrast.

Fig. 4

The effect of varying the size of the training set versus the error between automated and gold standard manual segmentations. Error is defined on the number of incorrectly classified voxels inside the bounding box. Values are obtained for 5, 10, 15, 20, 25, and 30 brains. Note that the curves level off after 20 brains indicating diminishing returns by using more than 20 brain on which to train.

Fig. 5

Significance maps (p-maps) based on manual, Ada-SVM, AdaBoost, and FreeSurfer segmentations

Fig. 6

Cumulative distribution of p-values for different methods. (a) shows the p-values when the covariate is the Alzheimer’s disease diagnosis. (b) shows the p-values when the covariate is the MMSE [17] score. These CDF plots are commonly generated when using false discovery rate methods to assign overall significance values to statistical maps [7], [18], [52]; they may also be used to compare effect sizes of different methods, subject to certain caveats [29], as they show the proportion of supra-threshold voxels in a statistical map, for a range of thresholds. A cumulative plot of p-values in a statistical map, after the p-values have been sorted into numerical order, can compare the proportion of supra-threshold statistics with null data, or between one method and another, to assess their power to detect statistical differences that survive thresholding at both weak and strict thresholds (in fact at any threshold in the range [0,1]). In the examples shown here, the cumulative distribution function of the p-values observed for the statistical comparison of patients versus controls is plotted against the corresponding p-value that would be expected, under the null hypothesis of no group difference (shown here in black).