A critical evaluation of the current "p-value controversy"

Abstract

This article has been triggered by the initiative launched in March 2016 by the Board of Directors of the American Statistical Association (ASA) to counteract the current p-value focus of statistical research practices that allegedly "have contributed to a reproducibility crisis in science." It is pointed out that in the very wide field of statistics applied to medicine, many of the problems raised in the ASA statement are not as severe as in the areas the authors may have primarily in mind, although several of them are well-known experts in biostatistics and epidemiology. This is mainly due to the fact that a large proportion of medical research falls under the realm of a well developed body of regulatory rules banning the most frequently occurring misuses of p-values. Furthermore, it is argued that reducing the statistical hypotheses tests nowadays available to the class of procedures based on p-values calculated under a traditional one-point null hypothesis amounts to ignoring important developments having taken place and going on within the statistical sciences. Although hypotheses testing is still an indispensable part of the statistical methodology required in medical and other areas of empirical research, there is a large repertoire of methods based on different paradigms of inference that provide ample options for supplementing and enhancing the methods of data analysis blamed in the ASA statement for causing a crisis.

Keywords: Bayesian inference; Data mining; Measures of evidence; Multiplicity correction; Prediction; Reproducibility of experiments.