Replication and innovation versus a perfect '.05'

Abstract

Slotnick (this issue) makes a strong case that any attempt to assess the reliabiltiy of statistical correction procedures should use truly random data. In addition, however, there is an important side effect of the over-reliance on any given threshold to determine the worth of an experiment. Placing too much faith in any method of correction obscures the point that replication across labs remains a most critical part of scientific study. Especially for expensive methods, such as fMRI, an overemphasis on increasingly conservative thresholds can negatively impact the potential for replication of studies and the pursuit and reporting of innovative results.

Keywords: fMRI; statistics; type II errors.