Inconsistencies between reported test statistics and p-values in two psychiatry journals

Abstract

A recent survey of the British Medical Journal (BMJ) and Nature revealed that inconsistencies in reported statistics were common. We sought to replicate that survey in the psychiatry literature. We checked the consistency of reported t-test, F-test and chi(2)-test values with their corresponding p-values in the 2005 issues of the Australian and New Zealand Journal of Psychiatry (ANZJP) and compared this with the issues of the ANZJP from 2000, and with a similar journal, Acta Psychiatrica Scandinavica (APS). A reported p-value was 'inconsistent' if it differed (at its reported number of decimal places) from our calculated p-values (using three different software packages), which we based on the reported test statistic and degrees of freedom. Of the 546 results that we checked, 78 (14.3%) of the p-values were inconsistent with the corresponding degrees of freedom and test statistic. Similar rates of inconsistency were found in APS and ANZJP, and when comparing the ANZJP between 2000 and 2005. The percentages of articles with at least one inconsistency were 8.5% for ANZJP 2005, 9.9% for ANZJP 2000 and 12.1% for APS. We conclude that inconsistencies in p-values are common and may reflect errors of analysis and rounding, typographic errors or typesetting errors. Suggestions for reducing the occurrence of such inconsistencies are provided.