Quality of flow diagram in systematic review and/or meta-analysis

Abstract

Systematic reviews and/or meta-analyses generally provide the best evidence for medical research. Authors are recommended to use flow diagrams to present the review process, allowing for better understanding among readers. However, no studies as of yet have assessed the quality of flow diagrams in systematic review/meta-analyses. Our study aims to evaluate the quality of systematic review/meta-analyses over a period of ten years, by assessing the quality of the flow diagrams, and the correlation to the methodological quality. Two hundred articles of "systematic review" and/or "meta-analysis" from January 2004 to August 2015 were randomly retrieved in Pubmed to be assessed for the flow diagram and methodological qualities. The flow diagrams were evaluated using a 16-grade scale corresponding to the four stages of PRISMA flow diagram. It composes four parts: Identification, Screening, Eligibility and Inclusion. Of the 200 articles screened, 154 articles were included and were assessed with AMSTAR checklist. Among them, 78 articles (50.6%) had the flow diagram. Over ten years, the proportion of papers with flow diagram available had been increasing significantly with regression coefficient beta = 5.649 (p = 0.002). However, the improvement in quality of the flow diagram increased slightly but not significantly (regression coefficient beta = 0.177, p = 0.133). Our analysis showed high variation in the proportion of articles that reported flow diagram components. The lowest proportions were 1% for reporting methods of duplicates removal in screening phase, followed by 6% for manual search in identification phase, 22% for number of studies for each specific/subgroup analysis, 27% for number of articles retrieved from each database, and 31% for number of studies included in qualitative analysis. The flow diagram quality was correlated with the methodological quality with the Pearson's coefficient r = 0.32 (p = 0.0039). Therefore, this review suggests that the reporting quality of flow diagram is less satisfactory, hence not maximizing the potential benefit of the flow diagrams. A guideline with standardized flow diagram is recommended to improve the quality of systematic reviews, and to enable better reader comprehension of the review process.

Fig 1. Diagram of selecting systematic reviews for the study.

Fig 2. The proportion of papers with flow diagram over ten years.

Fig 3. Characteristic of all items in flow diagram and the presence of each item.

Fig 4. Scatter plot of AMSTAR total score and flow diagram score.

Fig 5. Trend of AMSTAR score (A) and flow diagram score (B) over ten years.

Fig 6. Correlation of flow diagram score and impact factor of journals.

Fig 7. New flow diagram template proposed.