Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep 20;12(1):168.
doi: 10.1186/s13643-023-02316-z.

The relationship between methodological quality and the use of retracted publications in evidence syntheses

Caitlin J Bakker  1   2 , Nicole Theis-Mahon  3 , Sarah Jane Brown  3 , Maurice P Zeegers  4
Affiliations

The relationship between methodological quality and the use of retracted publications in evidence syntheses

Caitlin J Bakker et al. Syst Rev. .

Abstract

Background: Evidence syntheses cite retracted publications. However, citation is not necessarily endorsement, as authors may be criticizing or refuting its findings. We investigated the sentiment of these citations-whether they were critical or supportive-and associations with the methodological quality of the evidence synthesis, reason for the retraction, and time between publication and retraction.

Methods: Using a sample of 286 evidence syntheses containing 324 citations to retracted publications in the field of pharmacy, we used AMSTAR-2 to assess methodological quality. We used scite.ai and a human screener to determine citation sentiment. We conducted a Pearson's chi-square test to assess associations between citation sentiment, methodological quality, and reason for retraction, and one-way ANOVAs to investigate association between time, methodological quality, and citation sentiment.

Results: Almost 70% of the evidence syntheses in our sample were of critically low quality. We found that these critically low-quality evidence syntheses were more associated with positive statements while high-quality evidence syntheses were more associated with negative citation of retracted publications. In our sample of 324 citations, 20.4% of citations to retracted publications noted that the publication had been retracted.

Conclusion: The association between high-quality evidence syntheses and recognition of a publication's retracted status may indicate that best practices are sufficient. However, the volume of critically low-quality evidence syntheses ultimately perpetuates the citation of retracted publications with no indication of their retracted status. Strengthening journal requirements around the quality of evidence syntheses may lessen the inappropriate citation of retracted publications.

Keywords: Publication ethics; Research integrity; Retraction of publication; Scientific misconduct; Systematic reviews.

PubMed Disclaimer

Conflict of interest statement

Caitlin Bakker is a subject matter expert for the American Gastroenterological Association and has non-financial relationships with Cochrane Urology and National Information Standards Organization. Other authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Identification of evidence syntheses citing retracted publications
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Vuong QH, La VP, Ho MT, Vuong TT, Ho MT. Characteristics of retracted articles based on retraction data from online sources through February 2019. Sci Ed. 2020;7(1):34–44.
    1. Garfield E. What does automation of citation mean? In: Essays of an information scientist. Vol. 1. Philadelphia: ISI Press; 1970. p 98-99.
    1. Garfield E. Can citation indexing be automated? In: Stevens ME, Giuliano VE, Heilprin LB, editors. Statistical Association Methods for Mechanized Documentation, Symposium Proceedings, Washington, 1964. National Bureau of Standards Miscellaneous Publication, vol. 269. US Government Printing Office; 1965. p 189–92.
    1. Cozzens SE. What do citations count? Scientometrics. 1988;15:437–447.
    1. Merton RK. The sociology of science: theoretical and empirical investigations - Robert K. Merton - Google Books. Chicago, IL: University of Chicago Press; 1973. Available from: https://books.google.ca/books?hl=en&lr=&id=zPvcHuUMEMwC&oi=fnd&pg=PR9&ot.... Cited 2023 May 3.

Publication types

LinkOut - more resources