Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 1998 Nov;13(11):753-6.
doi: 10.1046/j.1525-1497.1998.00227.x.

Statistical reviewing policies of medical journals: caveat lector?

S N Goodman  1 , D G Altman, S L George
Affiliations

Statistical reviewing policies of medical journals: caveat lector?

S N Goodman et al. J Gen Intern Med. 1998 Nov.

Abstract

Objective: To describe the current policies regarding statistical review of clinical research in biomedical journals.

Design: Cross-sectional survey.

Participants: Editors of biomedical journals that publish original clinical research.

Measurements: General policies on statistical review, types of persons used for statistical reviewing, compensation of statistical reviewers, percentage of articles subject to such review, percentage of time statistical review makes an important difference, journal circulation, and selectivity.

Main results: Of 171 journals, 114 (67%) responded to the survey. About one third of journals had policies that guaranteed statistical review for all accepted manuscripts. In approximately half of the journals, articles were sent for statistical review at the discretion of the editor. There was some evidence that statistical review policies differed between journals of different circulation size. In journals in the top quartile of circulation (> 25,000) the probability of definitely having a statistical review before an acceptance decision was 52%, but it was only 27% in journals in the lower three quartiles (p = .09). The probability of a statistical consultant on staff ranged from 31% in the bottom quarter, to 58% in the middle two, to 82% in the highest quarter (p < .001). Editors judged that statistical review resulted in an important change in a manuscript about half of the time.

Conclusions: Except in the largest circulation medical journals, the probability of formal methodologic review of original clinical research is fairly low. As readers and researchers depend on the journals to assess the validity of the statistical methods and logic used in published reports, this is potentially a serious problem. This situation may exist because the cost of such statistical review can be considerable, and because finding appropriate reviewers can be difficult. It may also exist partly because editors or publishers may not regard such review as important. The professions of medical publishing, statistics, epidemiology, and other quantitative disciplines should work together to address this problem.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Percentage of submitted research that is published, by the circulation of the journal. The spacing on the circulation axis is logarithmic. Linear regression performed without the outlier in the upper right showed a 5.3% reduction (95% confidence interval 4.1%, 6.5%) in probability of acceptance with each doubling of circulation (p < .001
See this image and copyright information in PMC

References

    1. Altman DG, Bland JM. Improving doctors' understanding of statistics. J R Stat Soc A. 1991;154:223–67.
    1. Weiss ST, Samet JM. An assessment of physician knowledge of epidemiology and biostatistics. J Med Educ. 1980;55:692–7. - PubMed
    1. Wulff HR, Andersen B, Brandenhoff P, Guttler F. What do doctors know about statistics? Stat Med. 1987;6:3–10. - PubMed
    1. Borak J, Veiullieux S. Errors of intuitive logic among physicians. Soc Sci Med. 1982;16:1939–47. - PubMed
    1. Friedman SB, Phillips S. What's the difference? Pediatric residents and their inaccurate concepts regarding statistics. Pediatrics. 1981;68:644–6. - PubMed