Multiple Testing and Protection Against a Type 1 (False Positive) Error Using the Bonferroni and Hochberg Corrections

Abstract

In a given study, if many related outcomes are tested for statistical significance, one or more outcomes may emerge significant at the P < 0.05 level not because they are truly significant in the population but because of chance. The larger the number of statistical tests performed, the greater the risk that some of the significant findings are significant because of chance. There are many ways to protect against such false positive or Type 1 errors. The simplest way is to set a more stringent threshold for statistical significance than P < 0.05. This can be done using either the Bonferroni or the Hochberg correction. Using the Bonferroni correction, 0.05 is divided by the number of statistical tests being performed and the result is set as the critical P value for statistical significance. Using the Hochberg correction, the P values obtained from the different statistical tests are arranged in descending order of magnitude, and each P value is assessed for significance against progressively more stringent levels for significance. The Bonferroni and Hochberg procedures are explained with the help of examples.

Keywords: Bonferroni correction; Hochberg correction; P value; false positive error; multiple testing; type 1 error.