Reproducibility of Brain Morphometry from Short-Term Repeat Clinical MRI Examinations: A Retrospective Study

Abstract

Purpose: To assess the inter session reproducibility of automatic segmented MRI-derived measures by FreeSurfer in a group of subjects with normal-appearing MR images.

Materials and methods: After retrospectively reviewing a brain MRI database from our institute consisting of 14,758 adults, those subjects who had repeat scans and had no history of neurodegenerative disorders were selected for morphometry analysis using FreeSurfer. A total of 34 subjects were grouped by MRI scanner model. After automatic segmentation using FreeSurfer, label-wise comparison (involving area, thickness, and volume) was performed on all segmented results. An intraclass correlation coefficient was used to estimate the agreement between sessions. Wilcoxon signed rank test was used to assess the population mean rank differences across sessions. Mean-difference analysis was used to evaluate the difference intervals across scanners. Absolute percent difference was used to estimate the reproducibility errors across the MRI models. Kruskal-Wallis test was used to determine the across-scanner effect.

Results: The agreement in segmentation results for area, volume, and thickness measurements of all segmented anatomical labels was generally higher in Signa Excite and Verio models when compared with Sonata and TrioTim models. There were significant rank differences found across sessions in some labels of different measures. Smaller difference intervals in global volume measurements were noted on images acquired by Signa Excite and Verio models. For some brain regions, significant MRI model effects were observed on certain segmentation results.

Conclusions: Short-term scan-rescan reliability of automatic brain MRI morphometry is feasible in the clinical setting. However, since repeatability of software performance is contingent on the reproducibility of the scanner performance, the scanner performance must be calibrated before conducting such studies or before using such software for retrospective reviewing.

Fig 1. The violin plot of label-wise agreement (ICC) for each MRI scan.

The bootstrapped ICCs of each label from each measurement produced by FreeSurfer are summarized in Fig 1. In addition to the information in the box plot, the shape of the distribution visualized in the violin plot helps to detect clusters or bumps within a distribution. This plot reflects the density of label-wise ICC among different scanners grouped by measurement and parcellation/segmentation types. The distribution of the density plot reveals the portion from different levels that are in agreement within each group. Abbreviations: a2009s, Destrieux Atlas; aparc, Desikan-Killiany Atlas; wm, subcortical white matter; aseg, miscellaneous structures.

Fig 2. The violin plot of the p value of label-wise Wilcoxon signed rank test for each MRI scan.

The p value is presented as log₁₀(p). This plot reflects the distribution of label-wise p values between different machines by different measurement type and parcellation/segmentation type. The number and percentage of the regions of statistical significance (p < 0.01 or log₁₀(p) < -2) were also labeled below the violin distribution. Abbreviations: a2009s, Destrieux Atlas; aparc, Desikan-Killiany Atlas; wm, subcortical white matter; aseg, miscellaneous structures.

Fig 3. The regions of significant difference using Wilcoxon signed rank test are labeled and colored on the cortical surface for visualization.

Fig 4. Results of the Bland-Altman analysis comparing results obtained from repeat MRI scans using different MRI models and selected global measures of brain volume.

These plots represent the difference between repeat MRI scans against the average value for each pair of measurements. A continuous line and two dotted lines represent the mean difference between each pair of measurements and the limits of agreement, respectively, for each BA plot.