A posteriori model validation for the temporal order of directed functional connectivity maps

Abstract

A posteriori model validation for the temporal order of neural directed functional connectivity maps is rare. This is striking because models that require sequential independence among residuals are regularly implemented. The aim of the current study was (a) to apply to directed functional connectivity maps of functional magnetic resonance imaging data an a posteriori model validation procedure (i.e., white noise tests of one-step-ahead prediction errors combined with decision criteria for revising the maps based upon Lagrange Multiplier tests), and (b) to demonstrate how the procedure applies to single-subject simulated, single-subject task-related, and multi-subject resting state data. Directed functional connectivity was determined by the unified structural equation model family of approaches in order to map contemporaneous and first order lagged connections among brain regions at the group- and individual-levels while incorporating external input, then white noise tests were run. Findings revealed that the validation procedure successfully detected unmodeled sequential dependencies among residuals and recovered higher order (greater than one) simulated connections, and that the procedure can accommodate task-related input. Findings also revealed that lags greater than one were present in resting state data: With a group-level network that contained only contemporaneous and first order connections, 44% of subjects required second order, individual-level connections in order to obtain maps with white noise residuals. Results have broad methodological relevance (e.g., temporal validation is necessary after directed functional connectivity analyses because the presence of unmodeled higher order sequential dependencies may bias parameter estimates) and substantive implications (e.g., higher order lags may be common in resting state data).

Keywords: a posteriori model validation; directed functional connectivity; neuroimaging; structural vector autoregression; temporal order; unified structural equation modeling.

Figure 1

Single-subject simulation showing how the a posteriori model validation procedure can be applied to fMRI data analyzed for directed functional connectivity using the uSEM family of approaches. Dashed lines reflect first order connections, dotted lines reflect second order connections, solid lines reflect contemporaneous connections, and values show connection strengths (i.e., beta-weights; all are significant at p = 0.05). (A) Simulation for a single subject according to a second order uSEM with 3 ROIs and 200 time points, with beta-weights showing true parameter values. (B) Map resulting from a first order data-driven uSEM that did not contain white noise residuals. (C) Map resulting from a second order data-driven uSEM that did not contain white noise residuals. (D) Final map with white noise residuals resulting from the freeing of two additional confirmatory parameters, as indicated by modification indices from the second order data-driven uSEM white noise test. The final map precisely recovered the simulated map, with true parameter values falling within the bracketed 95% confidence intervals of the estimated beta-weights. See Table 1 for model fit and white noise test results.

Figure 2

A posteriori model validation procedure for the temporal order of directed functional connectivity maps generated by the uSEM family of approaches (i.e., uSEM, euSEM, GIMME), consisting of white noise testing of prediction errors and the application of decision criteria for generating higher order individual-level maps, when necessary.

Figure 3

Directed functional connectivity map from euSEM analysis of single-subject empirical data with external input (i.e., a vector indicating the experimental condition of a verbal working memory task). Dashed lines reflect first order connections, solid lines reflect contemporaneous connections, arrows reflect ROI connections, circular endpoints reflect direct effects of the task, diamond endpoints reflect modulating effects of the task, and values show connection strengths (i.e., beta-weights; all are significant at p = 0.05). The map fit the data well and had white noise residuals; see fit statistics in the text. ACC, anterior cingulate cortex; R/L DLPFC, right/left dorsolateral prefrontal cortex; R/L LPM, right/left lateral premotor cortex; R/L IPL, right/left inferior parietal lobule.

Figure 4

Exemplar final directed functional connectivity maps from GIMME analysis and a posteriori model validation of a 32-subject resting state data set. Thick lines reflect group-level connections, thin lines reflect individual-level connections, dashed lines reflect first order connections, dotted lines reflect second order connections, solid lines reflect contemporaneous connections, and values show connection strengths (i.e., beta-weights; all are significant at p = 0.05 except for group paths ≤ |0.13|). (A) Final map for a subject requiring only the lag one and contemporaneous connections estimated in GIMME; the map fit the data well [χ²₍₁₂₎ = 17.40, p > 0.05, RMSEA = 0.05, SRMR = 0.04, CFI = 0.99, NNFI = 0.98] and contained white noise residuals [χ²₍₁₂₆₎ = 136.94, p > 0.05, RMSEA = 0.00, SRMR = 0.10, CFI = 0.98, NNFI = 0.98]. (B) Final map for a subject requiring second order connections, determined via a confirmatory second order uSEM, which fit the data well [χ²₍₂₆₎ = 54.54, p < 0.001, RMSEA = 0.08, SRMR = 0.04, CFI = 0.98, NNFI = 0.96] and contained white noise residuals [χ²₍₁₂₆₎ = 135.08, p > 0.05, RMSEA = 0.00, SRMR = 0.07, CFI = 0.98, NNFI = 0.98]. (C) Final map for a subject requiring second order connections, determined via a data-driven second order uSEM, which fit the data well [χ²₍₂₃₎ = 17.35, p > 0.05, RMSEA = 0.00, SRMR = 0.02, CFI = 1.00, NNFI = 1.00] and contained white noise residuals [χ²₍₁₂₆₎ = 72.64, p > 0.05, RMSEA = 0.00, SRMR = 0.06, CFI = 1.00, NNFI = 1.00]. PCC, posterior cingulate cortex; MPFC, medial prefrontal cortex; R/L LP, right/left lateral parietal lobule.