Evidence of reporting biases in voxel-based morphometry (VBM) studies of psychiatric and neurological disorders

Abstract

Objectives: To evaluate whether biases may influence the findings of whole-brain structural imaging literature.

Methods: Forty-seven whole-brain voxel-based meta-analyses including voxel-based morphometry (VBM) studies in neuropsychiatric conditions were included, for a total of 324 individual VBM studies. The total sample size, the overall number of foci, and different moderators were extracted both at the level of the individual studies and at the level of the meta-analyses.

Results: Sample size ranged from 12 to 545 (median n = 47) per VBM study. The median number of reported foci per study was six. VBM studies with larger sample sizes reported only slightly more abnormalities than smaller studies (2% increase in the number of foci per 10-patients increase in sample size). A similar pattern was seen in several analyses according to different moderator variables with some possible modulating evidence for the statistical threshold employed, publication year and number of coauthors. Whole-brain meta-analyses (median sample size n = 534) found fewer foci (median = 3) than single studies and overall they showed no significant increase in the number of foci with increasing sample size. Meta-analyses with ≥10 VBM studies reported a median of three foci and showed a significant increase with increasing sample size, while there was no relationship between sample size and number of foci (median = 5) in meta-analyses with <10 VBM studies.

Conclusions: The number of foci reported in small VBM studies and even in meta-analyses with few studies may often be inflated. This picture is consistent with reporting biases affecting small studies.

Keywords: VBM; bias; dementia; neuroimaging; psychosis; structural.

Figure 1

Expected relationship between sample size and number of abnormalities detected by whole‐brain structural neuroimaging techniques. Footnote: The number of abnormalities correctly detected as abnormal (P < 0.05) in a simulated study with that sample size is shown with solid lines. The number falsely detected abnormalities is shown with dotted lines. In all simulations, brain was composed of 5,000 independent parts, of which 95% were normal and 5% were abnormal.

Figure 2

Relationship between sample size and number of clusters in simulated VBM data.

Figure 3

PRISMA Flow chart of literature search.

Figure 4

Relationship between sample size and identified number of foci with abnormalities.

Figure 5

Relationship between number of authors and number of identified foci.

Figure 6

Relationship between sample size and number of identified foci in studies using FDR correction for multiple comparisons and those using FWE correction for multiple comparisons. Difference in regression slope P = 0.005.

Figure 7

Relationship between sample size and number of identified foci in all meta‐analyses (above), meta‐analyses with at least 10 studies (bottom left) and meta‐analyses with less than 10 studies (bottom right).