Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Sep 25;5(1):20230041.
doi: 10.1259/bjro.20230041. eCollection 2023.

A plea for more careful scholarship in reviewing evidence: the case of mammographic screening

Stephen W Duffy  1 , Laszlo Tabar  2 , Tony H H Chen  3 , Amy M F Yen  4 , Peter B Dean  5 , Robert A Smith  6
Affiliations
Review

A plea for more careful scholarship in reviewing evidence: the case of mammographic screening

Stephen W Duffy et al. BJR Open. .

Abstract

Objectives: To identify issues of principle and practice giving rise to misunderstandings in reviewing evidence, to illustrate these by reference to the Nordic Cochrane Review (NCR) and its interpretation of two trials of mammographic screening, and to draw lessons for future reviewing of published results.

Methods: A narrative review of the publications of the Nordic Cochrane Review of mammographic screening (NCR), the Swedish Two-County Trial (S2C) and the Canadian National Breast Screening Study 1 and 2 (CNBSS-1 and CNBSS-2).

Results: The NCR concluded that the S2C was unreliable, despite the review's complaints being shown to be mistaken, by direct reference to the original primary publications of the S2C. Repeated concerns were expressed by others about potential subversion of randomisation in CNBSS-1 and CNBSS-2; however, the NCR continued to rely heavily on the results of these trials. Since 2022, however, eyewitness evidence of such subversion has been in the public domain.

Conclusions: An over-reliance on nominal satisfaction of checklists of criteria in systematic reviewing can lead to erroneous conclusions. This occurred in the case of the NCR, which concluded that mammographic screening was ineffective or minimally effective. Broader and more even-handed reviews of the evidence show that screening confers a substantial reduction in breast cancer mortality.

Advances in knowledge: Those carrying out systematic reviews should be aware of the dangers of over-reliance on checklists and guidelines. Readers of systematic reviews should be aware that a systematic review is just another study, with the capability that all studies have of coming to incorrect conclusions. When a review seems to overturn the current position, it is essential to revisit the publications of the primary research.

PubMed Disclaimer

References

    1. Nelson HD, Fu R, Cantor A, Pappas M, Daeges M, Humphrey L. Effectiveness of breast cancer screening: systematic review and meta-analysis to update the 2009 U.S. Ann Intern Med 2016; 164: 244–55. doi: 10.7326/M15-0969 - DOI - PubMed
    1. Lauby-Secretan B, Scoccianti C, Loomis D. International agency for research on cancer Handbook working group. breast-cancer Screening- viewpoint of the IARC working group. N Engl J Med 2015; 372: 2353–58. doi: 10.1056/NEJMsr1504363 - DOI - PubMed
    1. Schünemann HJ, Lerda D, Quinn C, Follmann M, Alonso-Coello P, Rossi PG, et al. . Breast cancer screening and diagnosis: A Synopsis of the European breast guidelines. Ann Intern Med 2020; 172: 46–56. doi: 10.7326/M19-2125 - DOI - PubMed
    1. Independent UK Panel on Breast Cancer Screening . The benefits and harms of breast cancer screening: an independent review. Lancet 2012; 380: 1778–86. doi: 10.1016/S0140-6736(12)61611-0 - DOI - PubMed
    1. Monticciolo DL, Newell MS, Hendrick RE, Helvie MA, Moy L, Monsees B, et al. . Breast cancer screening for average-risk women: recommendations from the ACR Commission on breast imaging. J Am Coll Radiol 2017; 14: 1137–43. doi: 10.1016/j.jacr.2017.06.001 - DOI - PubMed