Methods of reporting statistical results from medical research studies

Abstract

There has recently been vigorous debate about the most appropriate way to report statistical results from medical studies, particularly concerning the relative merits of significance tests, confidence intervals, or other methods such as likelihood. Arguing that p values have often been abused in the past, some biomedical journals now require authors to emphasize confidence intervals instead. A review of the debate reveals points in favor of each of the proposed methods, and more than one type of analysis may be required to obtain a full perspective on the data. Furthermore, a change in editorial policy alone is unlikely to eliminate future statistical errors. It therefore appears unsatisfactory and coercive of editors to mandate the exclusive use of any one approach, based on speculative estimates of the likely rate of its correct usage by future authors. Instead, further work is needed to better understand how consumers of medical research interpret studies, particularly how they are influenced by the authors' methods of summarizing the data. Until then, editorial flexibility is required, so that authors can provide readers with enough information to form their own conclusions from a given set of results. Evidence from papers recently published in the American Journal of Epidemiology suggests that authors are now tending to report results using both significance testing and estimation techniques. This seems entirely appropriate, to avoid the strictures implied by rigid adherence to only one method.