Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Oct;9(10):e01363.
doi: 10.1002/brb3.1363. Epub 2019 Sep 4.

Test-retest reproducibility of a multi-atlas automated segmentation tool on multimodality brain MRI

Thiago J R Rezende  1 , Brunno M Campos  1 , Johnny Hsu  2 , Yue Li  3 , Can Ceritoglu  4 , Kwame Kutten  4 , Marcondes C França Junior  1 , Susumu Mori  2 , Michael I Miller  4 , Andreia V Faria  2
Affiliations

Test-retest reproducibility of a multi-atlas automated segmentation tool on multimodality brain MRI

Thiago J R Rezende et al. Brain Behav. 2019 Oct.

Abstract

Introduction: The increasing use of large sample sizes for population and personalized medicine requires high-throughput tools for imaging processing that can handle large amounts of data with diverse image modalities, perform a biologically meaningful information reduction, and result in comprehensive quantification. Exploring the reproducibility of these tools reveals the specific strengths and weaknesses that heavily influence the interpretation of results, contributing to transparence in science.

Methods: We tested-retested the reproducibility of MRICloud, a free automated method for whole-brain, multimodal MRI segmentation and quantification, on two public, independent datasets of healthy adults.

Results: The reproducibility was extremely high for T1-volumetric analysis, high for diffusion tensor images (DTI) (however, regionally variable), and low for resting-state fMRI.

Conclusion: In general, the reproducibility of the different modalities was slightly superior to that of widely used software. This analysis serves as a normative reference for planning samples and for the interpretation of structure-based MRI studies.

Keywords: automated segmentation; multimodality brain MRI; reproducibility; test-retest.

PubMed Disclaimer

Conflict of interest statement

None declared.

Figures

Figure 1
Figure 1
I2C2 for the results of T1‐volumetric analysis, fractional anisotropy (FA), and mean diffusivity (MD) from DTI, and resting‐state fMRI, in two independent datasets (Kirby21 and HCP), using different platforms (MRICloud [MC], FreeSurfer [FS], Connectivity toolbox in SPM [CONN‐SPM])
Figure 2
Figure 2
Color‐coded regional ICCs for the volumetric outputs of MRICloud (MC) and FreeSurfer (FS), in two independent datasets (Kirby21 and HCP), overlaid on a representative brain
Figure 3
Figure 3
Top: 3D PCA plot created with the volumes of Kirby21 cortical areas, outputted by MRICloud (MC) and FreeSurfer (FS). Individuals are color‐coded, that is, the same color represents a “test–retest” pair. Bottom: matrix of ranked distance between individuals in the three‐dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test–retest pair was scored 1 (dark blue). Test–retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than him/herself in the second scan) are framed in red
Figure 4
Figure 4
Top: 3D PCA plot created with the volumes of the Kirby21 deep gray matter areas, outputted by MRICLoud (MC) and FreeSurfer (FS). Individuals are color‐coded; that is, the same color represents a “test–retest” pair. Bottom: matrix of ranked distance between individuals in the three‐dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test–retest pair was scored 1 (dark blue). Test–retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/herself in the second scan) are framed in red
Figure 5
Figure 5
Color‐coded regional ICCs for the DTI outputs of MRICloud (FA, fractional anisotropy, MD, mean diffusivity) in two independent datasets (Kirby21 and HCP), overlaid on a representative brain
Figure 6
Figure 6
Top: 3D PCA plot created with the Kirby21 regional measures of fractional anisotropy (FA) and mean diffusivity (MD). Individuals were color‐coded; that is, the same color represents a “test–retest” pair. Bottom: matrix of ranked distance between individuals in the three‐dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test–retest pair scored 1 (dark blue). Test–retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/herself in the second scan) are framed in red
Figure 7
Figure 7
Color‐coded regional mean ICCs for the resting‐state fMRI outputs of MRICloud (MC) and CONN‐SPM, in two independent datasets (Kirby21 and HCP), overlaid on a representative brain, overlaid on a representative brain
Figure 8
Figure 8
Top: 3D PCA plot created with z‐transformed correlations between the Kirby21 fMRI time courses of a pair of seeds, outputted by MRICloud (MC) and SPM CONN. Individuals were color‐coded, that is, the same color represents a “test–retest” pair. Bottom: matrix of ranked distance between individuals in the three‐dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test–retest pair scored 1 (dark blue). Test–retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/herself in the second scan) are framed in red
Figure 9
Figure 9
Segmentation of cortex and white matter outputted from MRICloud (left) and FreeSurfer (right) of a brain with large degree of atrophy
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Airan, R. D. , Vogelstein, J. T. , Pillai, J. J. , Caffo, B. , Pekar, J. J. , & Sair, H. I. (2016). Factors affecting characterization and localization of interindividual differences in functional connectivity using MRI. Human Brain Mapping, 37, 1986–1997. 10.1002/hbm.23150 - DOI - PMC - PubMed
    1. Alexander, A. L. , Lee, J. E. , Wu, Y. C. , & Field, A. S. (2006). Comparison of diffusion tensor imaging measurements at 3.0 T versus 1.5 T with and without parallel imaging. Neuroimaging Clinics of North America, 16, 299–309. - PubMed
    1. Andellini, M. , Cannata, V. , Gazzellini, S. , Bernardi, B. , & Napolitano, A. (2015). Test‐retest reliability of graph metrics of resting state MRI functional brain networks: A review. Journal of Neuroscience Methods, 253, 183–192. 10.1016/j.jneumeth.2015.05.020 - DOI - PubMed
    1. Avants, B. B. , Tustison, N. J. , Song, G. , Cook, P. A. , Klein, A. , & Gee, J. C. (2011). A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54, 2033–2044. 10.1016/j.neuroimage.2010.09.025 - DOI - PMC - PubMed
    1. Behzadi, Y. , Restom, K. , Liau, J. , & Liu, T. T. (2007). A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. NeuroImage, 37, 90–101. - PMC - PubMed

Publication types