On a 'failed' attempt to manipulate visual metacognition with transcranial magnetic stimulation to prefrontal cortex

Abstract

Rounis, Maniscalco, Rothwell, Passingham, and Lau (2010) reported that stimulation of prefrontal cortex impairs visual metacognition. Bor, Schwartzman, Barrett, and Seth (2017) attempted to replicate this result, but adopted an experimental design that reduced their chanceof obtaining positive findings. Despite that, their results appeared initially consistent with those of Rounis et al., but they subsequently claimed it was necessary to discard ∼30% of their subjects, after which they reported a null result. Using computer simulations, we found that, contrary to their supposed purpose, excluding subjects by Bor et al.'s criteria does not reduce false positive rates. Including both their positive and negative result in a Bayesian framework, we show the correct interpretation is that PFC stimulation likely impaired visual metacognition, exactly contradicting Bor et al.'s claims. That lesion and inactivation studies demonstrate similar positive effects further suggests that Bor et al.'s reported negative finding isn't evidence against the role of prefrontal cortex in metacognition.

Keywords: Consciousness; Metacognition; Prefrontal cortex; Transcranial magnetic stimulation; Visual awareness.

Figure 1. Signal detection theoretic framework for the simulated spatial 2AFC task

For a given subject s, each stimulus presentation (either ◼◆or ◆◼) caused an internal response value (x), with X_◼◆ ~ N(d’/2, 1) and X_◆◼ ~ N(-d’/2, 1), and the subject then indicated which of the two shape configurations appeared. If x exceeds the subjects Type 1 criterion (c_s,1), then the subject responded “◼◆;” otherwise the subject responded “◆◼.” Objective performance capacity (d’) is the normalized distance between the two distributions. The subject then indicated how clearly/confidently they saw the stimulus, based on a comparison between x and the Type 2 criterion (-c_s,2 and c_s,2). If x < c_s,1 or x > c_s,2 the subject responded “clear”; otherwise the subject responded “unclear.” If TMS is present and assumed to degrade metacognitive sensitivity, noise (σ_s,TMS) is added for the Type 2 responses, such that the relevant computations are whether x < c_s,1 + ε_trial or x > c_s,2 + ε_trial, with ε_trial ~ N(0,σ_s,TMS) and resampled on each trial (see Supplementary Materials for more details).

Figure 2. Excluding subjects yields little impact on statistical power or false positive rates

As in the actual empirical studies, the effect of TMS was assessed by the statistical significance of the interaction between TMS (DLPFC or control) and time (before TMS and after TMS). (a) Excluding versus not excluding subjects yielded no meaningful change in false positive rate for the between-subjects design (Bor’s Exp. 1). Exclusion led to a small decrease in false positive rate for the double-repeat design (Bor’s Exp. 2), but this design inflated false positive rate in comparison to the between-subjects design (Bor Exp. 1) – although the inflation disappeared when we used permutation tests to address violations of normality assumptions (see main text). (b) Excluding subjects yielded an increase in statistical power compared with not excluding subjects for the between-subjects design (Bor’s Exp. 1), and the magnitude of this increase was even greater when using nonparametric permutation tests to evaluate statistical significance (see main text). Conversely, exclusion resulted in a decrease in power in comparison with no exclusion for the atypical double-repeat within-subjects design (Bor’s Exp. 2). Importantly, the double-repeat design (Bor’s Exp. 2) led to considerably lower rather than the intended higher statistical power when excluding subjects.