Improving in vivo human cerebral cortical surface reconstruction using data-driven super-resolution

Abstract

Accurate and automated reconstruction of the in vivo human cerebral cortical surface from anatomical magnetic resonance (MR) images facilitates the quantitative analysis of cortical structure. Anatomical MR images with sub-millimeter isotropic spatial resolution improve the accuracy of cortical surface and thickness estimation compared to the standard 1-millimeter isotropic resolution. Nonetheless, sub-millimeter resolution acquisitions require averaging multiple repetitions to achieve sufficient signal-to-noise ratio and are therefore long and potentially vulnerable to subject motion. We address this challenge by synthesizing sub-millimeter resolution images from standard 1-millimeter isotropic resolution images using a data-driven supervised machine learning-based super-resolution approach achieved via a deep convolutional neural network. We systematically characterize our approach using a large-scale simulated dataset and demonstrate its efficacy in empirical data. The super-resolution data provide improved cortical surfaces similar to those obtained from native sub-millimeter resolution data. The whole-brain mean absolute discrepancy in cortical surface positioning and thickness estimation is below 100 μm at the single-subject level and below 50 μm at the group level for the simulated data, and below 200 μm at the single-subject level and below 100 μm at the group level for the empirical data, making the accuracy of cortical surfaces derived from super-resolution sufficient for most applications.

Keywords: anatomical magnetic resonance imaging; convolutional neural network; cortical surface reconstruction; deep learning; super-resolution.

Figure 1

Image similarity. A representative axial slice from native 0.7-mm isotropic high-resolution (a), simulated 1-mm isotropic standard-resolution (b), up-sampled 0.7-mm isotropic high-resolution (c) and synthesized 0.7-mm isotropic high-resolution (d) T₁-weighted images from the HCP data. Insets (a–d): magnified views of visual cortex, with arrowheads highlighting locations where the synthesized images provide increased gray–white contrast. The difference maps (e, f) and the structural similarity index (SSIM) maps (g, h) depict the similarity between the up-sampled high-resolution (e, g) and the synthesized high-resolution (f, h) images compared to the native high-resolution images. SSIM scores range between 0 and 1, with larger values indicating higher similarity.

Figure 2

Gray–white contrast. Left-hemispheric vertex-wise contrast between the gray matter and white matter image intensity (expressed as [white—gray]/[white + gray]·100%) from native high-resolution (column i), simulated standard-resolution (column ii), up-sampled high-resolution (column iii) and synthesized high-resolution (column iv) T₁-weighted images of a representative subject (rows a–c) and averaged across 20 evaluation subjects (rows d–f) from the HCP data, displayed on inflated surface representations.

Figure 3

Cortical surfaces. Gray–white interface surfaces (a–d) and gray–CSF interface surfaces (e–h) reconstructed from native high-resolution (a, e), simulated standard-resolution (b, f), up-sampled high-resolution (c, g) and synthesized high-resolution (d, h) T₁-weighted images from a representative subject from the HCP data.

Figure 4

Cortical surface cross-sections. Enlarged views of an axial (rows a, b) and sagittal (rows c, d) image slice from the native high-resolution (columns i, ii), simulated standard-resolution (column iii), up-sampled high-resolution (column iv) and synthesized high-resolution (column v) T₁-weighted images near the central sulcus (rows a, b) and the calcarine sulcus (rows c, d). Cross-sections of the gray–white (rows a, c) and gray–CSF surfaces (rows b, d) reconstructed from respective images are visualized as colored contours (columns ii–v) and overlaid together on top of native high-resolution images for comparison (column i). The arrow heads (column i) highlight locations where the synthesized images provide clear improvement in the cortical surface estimates.

Figure 5

Estimation discrepancy. Left-hemispheric vertex-wise displacement/difference of the gray–white surfaces (rows a, d), gray–CSF surfaces (rows b, e) and cortical thickness (rows c, f) estimated from the up-sampled high-resolution and native high-resolution images (columns i–iii) and the synthesized and native high-resolution images (columns iv–vi) of a representative subject (rows a–c) and averaged across 20 evaluation subjects (rows d–f) from the HCP data, displayed on inflated surface representations. Different cortical regions from the FreeSurfer cortical parcellation (i.e., aparc.annot) are depicted as colored outlines.

Figure 6

Image similarity. A representative axial slice from native 0.75-mm isotropic high-resolution (a), 1-mm isotropic standard-resolution (b), up-sampled 0.75-mm isotropic high-resolution (c) and synthesized 0.75-mm isotropic high-resolution (d) T₁-weighted images from the MGH 7-Tesla data. Insets (a–d): magnified views of visual cortex, with arrowheads highlighting locations where the synthesized images provide increased gray–white contrast. The difference maps (e, f) and the structural similarity index (SSIM) maps (g, h) depict the similarity between the up-sampled high-resolution (e, g) and the synthesized high-resolution (f, h) images compared to the native high-resolution images. SSIM scores range between 0 and 1, with larger values indicating higher similarity.

Figure 7

Cortical surface cross-sections. Enlarged views of an axial (rows a, b) and sagittal (rows c, d) image slice from the native high-resolution (columns i, ii), standard-resolution (column iii), up-sampled high-resolution (column iv) and synthesized high-resolution (column v) T₁-weighted images near the central sulcus (rows a, b) and the calcarine sulcus (rows c, d) from the MGH 7-Tesla data. Cross-sections of the gray–white (rows a, c) and gray–CSF surfaces (rows b, d) reconstructed from respective images are visualized as colored contours (columns ii–v) and overlaid together on top of native high-resolution images for comparison (column i). The arrow heads (column i) highlight locations where the synthesized images provide clear improvement in the cortical surface estimates.

Figure 8

Network generalization. A representative axial slice from native high-resolution (a), standard-resolution (b), up-sampled high-resolution (c) and synthesized high-resolution (d, e) T₁-weighted images from the MGH 3-Tesla data. The synthesized images were obtained by directly applying the VDSR network trained and validated on T₁-weighted images of 4 subjects from the MGH 7-Tesla data (d) and a VDSR network initiated with parameters learned from the MGH 7-Tesla data and then fine-tuned using the data of one subject from the MGH 3-Tesla data. Cross-sections of the gray–white (f) and gray–CSF (g) surfaces reconstructed from respective images near the central sulcus are visualized as colored contours and overlaid together on top of native high-resolution images for comparison. The surface cross-sections from the directly synthesized images (purple contours) considerably overlap with and are therefore covered by the surface cross-sections from the synthesized images using the fine-tuned VDSR network (green contours). Each surface cross-section is individually displayed in Supplementary Fig. 8.