Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed

Abstract

Objective: Many reviews specifically aimed to assess the quality of randomized controlled trials (RCTs). We evaluated the quality of reporting in such reviews.

Study design and setting: PubMed and the Cochrane library were searched for all reviews assessing the quality of RCTs between 1987 and 2007, and experts in the field were also contacted.

Results: We found 177 reviews published from 1987 to 2007, 58% of which were published after 2002. Of these, 131 (74%) focused on the quality of RCTs, 44 (25%) on quality of reporting, and 2 (1%) assessed both. The search strategy was well reported (92%). The criteria for assessment were reported in 97% of the reviews but were defined in only 38%. Seventy-four different items and 26 different scales were identified. Allocation sequence generation and concealment were reported in 41% and 40%, respectively, but their adequacy was assessed in 20% and 29%, respectively; scales were used in 40% and Consolidated Standards of Reporting Trials (CONSORT) checklist in 12%.

Conclusion: The number of methodological reviews has dramatically increased in recent years. Despite an improved reporting of the methodology, how quality is assessed still raises important issues. Heterogeneity of criteria used and lack of definition may limit the relevance of these reviews.