Review of null hypothesis significance testing in the ophthalmic literature: are most 'significant' P values false positives?

Abstract

P values associated with null hypothesis significance testing (NHST) are almost universal in the ophthalmic literature. A P value < 0.05 is traditionally considered 'significant'. This concept may deflect further thought about the veracity of the results. P values influence the publishability of the data and have flow-on effects for funding success and the direction of future research. Despite their importance, the problems inherent in P values have been recognized since their inception, and in more recent years have been increasingly highlighted in some scientific fields. In this review, we aim to bring the problems associated with P values and NHST to the attention of the ophthalmic research community. We do not offer a universal solution to the problem of determining the veracity of a scientific claim; however, we demonstrate the need for caution in interpreting 'significant' P values by performing a Bayesian re-analysis of t-tests in the ophthalmic literature.

Keywords: biostatistics; epidemiology; journal.