Unbiased comparison of sample size estimates from longitudinal structural measures in ADNI

Abstract

Structural changes in neuroanatomical subregions can be measured using serial magnetic resonance imaging scans, and provide powerful biomarkers for detecting and monitoring Alzheimer's disease. The Alzheimer's Disease Neuroimaging Initiative (ADNI) has made a large database of longitudinal scans available, with one of its primary goals being to explore the utility of structural change measures for assessing treatment effects in clinical trials of putative disease-modifying therapies. Several ADNI-funded research laboratories have calculated such measures from the ADNI database and made their results publicly available. Here, using sample size estimates, we present a comparative analysis of the overall results that come from the application of each laboratory's extensive processing stream to the ADNI database. Obtaining accurate measures of change requires correcting for potential bias due to the measurement methods themselves; and obtaining realistic sample size estimates for treatment response, based on longitudinal imaging measures from natural history studies such as ADNI, requires calibrating measured change in patient cohorts with respect to longitudinal anatomical changes inherent to normal aging. We present results showing that significant longitudinal change is present in healthy control subjects who test negative for amyloid-β pathology. Therefore, sample size estimates as commonly reported from power calculations based on total structural change in patients, rather than change in patients relative to change in healthy controls, are likely to be unrealistically low for treatments targeting amyloid-related pathology. Of all the measures publicly available in ADNI, thinning of the entorhinal cortex quantified with the Quarc methodology provides the most powerful change biomarker.

Figure 1

Average cumulative atrophy for TBM Stat‐ROI, as a percentage of baseline volume, for HC, MCI, and AD subjects, along with linear fits for the additive bias estimate. The additive bias of 1.31% is large, equal to 68% of the change in the MCI cohort, and 57% of the change in the AD cohort, at 6‐months, and highly statistically significant (P = 0 to precision of Matlab numerical libraries). The linear fits are conservatively restricted to the 6‐ and 12‐month visits. Note the slight shift to the right for follow‐up visits. On average, actual visit dates occurred later than the nominal interval dates.

Figure 2

Annual rate of volume change in entorhinal cortex (ERC), hippocampus (Hipp), and whole brain (Brain and KN‐BSI), with 95% confidence intervals, calculated for all methodologies, along with the TBM Stat‐ROI. Aβ‐negative HCs are almost the same as all HCs, but have approximately a third to a half the change seen in MCI; Aβ‐positive HCs atrophy at a slightly higher rate than Aβ‐negative HCs. Thus, most HC change is not AD‐related; assuming it is leads to seriously underpowered clinical trials. All rates are corrected for bias, if any. Note that only a subset of all HCs had Aβ status determined.

Figure 3

Estimated sample sizes, per arm, to detect a 25% reduction in rate of change in MCI subjects, at the P < 0.05 level with 80% power assuming a 24‐month trial with scans every 6 months. Sample sizes are estimated using a linear mixed effects model with fixed intercepts (no relative change at baseline) and random slopes applied to all data available up through 36 months. Results for the conservative aging‐corrected rates of change are shown in red; results for the more realistic amyloid‐negative aging‐corrected rates of change are shown in green; and results for absolute (aging uncorrected) rates of change are shown in blue. Error bars show the 95% confidence intervals. All numerical values are shown Table II. P‐values for all head‐to‐head pairwise comparisons of measures are in Table IV. FS is FreeSurfer‐longitudinal. FreeSurfer‐cross‐sectional (FSx) generally performs poorer than FS; it is not shown here, but values for it are in Tables II and IV.

Figure 4

Estimated sample sizes, per arm, to detect a 25% reduction in rate of change in mild AD subjects, at the P < 0.05 level with 80% power assuming a 24‐month trial with scans every 6 months. See caption of Figure 3 for further details. Numerical values are shown in Table III. P‐values for all head‐to‐head pairwise comparisons of measures are in Table V.

Figure 5

Estimated sample sizes for MCI, as in Figure 3, but from pairwise head‐to‐head comparison of Quarc with: BSI (subjects in common: 287 MCI, 170 HC, 55 HC(Aβ⁻)); TBM (236 MCI, 132 HC, 41 HC(Aβ⁻)); FreeSurfer‐longitudinal (267 MCI, 161 HC, 49 HC(Aβ⁻)); and FreeSurfer‐cross‐sectional (305 MCI, 178 HC, 56 HC(Aβ⁻)). P‐values for all head‐to‐head pairwise comparisons of measures are in Table IV.

Figure 6

Estimated sample sizes for AD, as in Figure 4, but from pairwise head‐to‐head comparison of Quarc with BSI (123 AD subjects in common—see also Figure 5); TBM (97 AD); FreeSurfer‐longitudinal (FS; 107 AD); and FreeSurfer‐cross‐sectional (FSx; 129 AD; note different scale due to relatively poor performance for entorhinal cortex). P‐values for all head‐to‐head pairwise comparisons of measures are in Table V.