Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Dec 21:8:1268045.
doi: 10.3389/frma.2023.1268045. eCollection 2023.

Appraising systematic reviews: a comprehensive guide to ensuring validity and reliability

Nour Shaheen  1 , Ahmed Shaheen  1 , Alaa Ramadan  2 , Mahmoud Tarek Hefnawy  3   4 , Abdelraouf Ramadan  5 , Ismail A Ibrahim  6 , Maged Elsayed Hassanein  3   4 , Mohamed E Ashour  1 , Oliver Flouty  7
Affiliations
Review

Appraising systematic reviews: a comprehensive guide to ensuring validity and reliability

Nour Shaheen et al. Front Res Metr Anal. .

Abstract

Systematic reviews play a crucial role in evidence-based practices as they consolidate research findings to inform decision-making. However, it is essential to assess the quality of systematic reviews to prevent biased or inaccurate conclusions. This paper underscores the importance of adhering to recognized guidelines, such as the PRISMA statement and Cochrane Handbook. These recommendations advocate for systematic approaches and emphasize the documentation of critical components, including the search strategy and study selection. A thorough evaluation of methodologies, research quality, and overall evidence strength is essential during the appraisal process. Identifying potential sources of bias and review limitations, such as selective reporting or trial heterogeneity, is facilitated by tools like the Cochrane Risk of Bias and the AMSTAR 2 checklist. The assessment of included studies emphasizes formulating clear research questions and employing appropriate search strategies to construct robust reviews. Relevance and bias reduction are ensured through meticulous selection of inclusion and exclusion criteria. Accurate data synthesis, including appropriate data extraction and analysis, is necessary for drawing reliable conclusions. Meta-analysis, a statistical method for aggregating trial findings, improves the precision of treatment impact estimates. Systematic reviews should consider crucial factors such as addressing biases, disclosing conflicts of interest, and acknowledging review and methodological limitations. This paper aims to enhance the reliability of systematic reviews, ultimately improving decision-making in healthcare, public policy, and other domains. It provides academics, practitioners, and policymakers with a comprehensive understanding of the evaluation process, empowering them to make well-informed decisions based on robust data.

Keywords: bias evaluation; quality assessment; systematic review; systematic review appraisal; systematic review methodology.

PubMed Disclaimer

Conflict of interest statement

Disclosure is widely used as a way to manage a competing interest, which is a significant cause of bias in research. Taking into account the significance of systematic reviews and the differing incidence of conflicting interests in various research disciplines. For researchers, reviewers, and editors, the recognition and declaration of competing interests, particularly nonfinancial interests, continues to be difficult. To identify and reduce potential conflicting interests in systematic reviews, the International Council of Medical Journal Editors (ICMJE) must continue to create more effective and efficient tools (Yu et al., 2020).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Boolean operators; (A) describe using AND in the search strategy, (B) describe using OR.
See this image and copyright information in PMC

References

    1. Botes (2002). Concept analysis Some limitations and possible solutions. Curationis. 25, 23–27. 10.4102/curationis.v25i3.779 - DOI - PubMed
    1. Bramer W. M., De Jonge G. B., Rethlefsen M. L., Mast F., Kleijnen J. (2018). A systematic approach to searching an efficient and complete method to develop literature searches. JMLA. 106, 4. 10.5195/jmla.2018.283 - DOI - PMC - PubMed
    1. CASP (2023). CASP- Critical Appraisal Skills Programme. Available online at: https://casp-uk.net (accessed Nov 07, 2023).
    1. Chien W.-T., Norman I. (2009). The effectiveness and active ingredients of mutual support groups for family caregivers of people with psychotic disorders a literature review. Int. J. Nurs. Stud. 46, 1604–1623. 10.1016/j.ijnurstu.2009.04.003 - DOI - PubMed
    1. Cochrane Bias (2023). Rob 2 A Revised Cochrane Risk-of-Bias Tool for Randomized Trials. Available online at: https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-... (accessed Nov 07, 2023).

LinkOut - more resources