Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2021 Aug;29(4):203-215.
doi: 10.1080/10669817.2020.1839728. Epub 2020 Nov 17.

Should low-quality evidence dominate high-level evidence? A systematic review and meta-analysis of systematic reviews of musculoskeletal physical therapy interventions

Sean Riley  1 , Brian T Swanson  2 , Steven F Sawyer  3 , Jean-Michel Brismée  3 , Geoffrey Staysniak  1
Affiliations
Meta-Analysis

Should low-quality evidence dominate high-level evidence? A systematic review and meta-analysis of systematic reviews of musculoskeletal physical therapy interventions

Sean Riley et al. J Man Manip Ther. 2021 Aug.

Abstract

Objectives: To determine the: 1) quality of articles cited in systematic reviews (SRs); 2) methodological quality of the SRs; and 3) impact of quality on level 1A evidence.

Methods: SRs related to musculoskeletal physical therapy interventions were identified. The methodological quality of the SRs and articles cited by the SRs were assessed by two blinded reviewers. Data analysis was performed by a third blinded researcher. Additional comparisons were made based on the Journal Impact Factor, spin, financial bias, and conflict of interest.

Results: Twenty-four SRs were identified; 21/24 SRs had 'critically low' quality on the AMSTAR 2. Thirty-four percent of included studies were 'low quality,' and 58% of SRs included studies that had unreported external validity. One-half of the SRs represented 'spin,' and one-third of the SRs generated conclusions based on low-quality clinical trials.

Discussion: The 'critically low' SRs methodological quality was exacerbated by low-quality research inclusion. Most SRs failed to follow best practices, including prospective registration and integration of professional librarians in the search process. Based on the high proportion of SRs that include low-quality trials and overall low methodological quality, further discussion regarding practice recommendations on level vs. quality of evidence is warranted.

Level of evidence: 1a.

Keywords: Journal impact factor; bias; physical therapists; publications; randomized controlled trials as topic.

PubMed Disclaimer

Conflict of interest statement

No potential conflict of interest was reported by the authors.

Figures

Figure 1.
Figure 1.
PRISMA diagram
See this image and copyright information in PMC

References

    1. Burns PB, Rohrich RJ, Chung KC.. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011Jul;128(1):305–310. .PubMed PMID: 21701348; PubMed Central PMCID: PMCPMC3124652. - PMC - PubMed
    1. Zadro J, O’Keeffe M, Maher C. Do physical therapists follow evidence-based guidelines when managing musculoskeletal conditions? Systematic review. BMJ Open. 2019Oct7;9(10):e032329. PubMed PMID: 31591090; PubMed Central PMCID: PMCPMC6797428. - PMC - PubMed
    1. Mandel D, Littner Y, Mimouni FB, et al. Conclusiveness of the Cochrane neonatal reviews: a systematic analysis. Acta Paediatr. 2006Oct;95(10):1209–1212. PubMed PMID: 16982491. - PubMed
    1. Cohen S, Lubetzky R, Mimouni FB, et al. Conclusiveness of the Cochrane reviews in pediatric-gastroenterology: a systematic analysis. Eur J Gastroenterol Hepatol. 2013Feb;25(2):252–254. PubMed PMID: 23044810. - PubMed
    1. Cohen S, Mandel D, Mimouni FB, et al. Conclusiveness of the Cochrane reviews in nutrition: a systematic analysis. Eur J Clin Nutr. 2014Feb;68(2):143–145. PubMed PMID: 24327125. - PubMed