Poor Quality in Systematic Reviews on PTSD and EMDR - An Examination of Search Methodology and Reporting

Abstract

Background: Different user groups regard systematic reviews as reliable and valuable sources for answering research questions. For systematic reviews to fulfill their purpose, methodological quality in all stages are of importance. The studies identified in a systematic search form the basis of the review, thus the search process methodology is important for both performing and reporting the search. The purpose of the present study was to evaluate the quality of non-Cochrane systematic reviews by analyzing how they perform and report the search. This is exemplified by systematic reviews on eye movement desensitization and reprocessing (EMDR), a trauma-focused therapy commonly used for post-traumatic stress disorder (PTSD). Methods and Results: We examined the method chapters of 20 systematic reviews on the subject, and rated their searches and reporting using relevant elements from the Cochrane Handbook and PRISMA. We found inadequacies in the methods employed for searching and reporting the search strategy, which could have been avoided by greater adherence to guiding documents for performing systematic reviews. Conclusion: Our findings raise important questions for future debate on the risk of omitting studies, thus impairing the conclusions in a systematic review. For clinical purposes, researchers should investigate if, and how, the search strategy in a systematic review affects the body of knowledge and the results.

Keywords: Cochrane handbook; PRISMA; critical appraisal; search methodology; systematic reviews.

FIGURE 1

Prisma2009 flow diagram. Adapted from Moher et al. (2009).

FIGURE 2

Number of reviews using selected databases.

FIGURE 3

Use of PRISMA items. Number of articles reviewed adhering to PRISMA guidelines regarding the reporting of Title, Information sources, Search and Flow diagram.