Opposing systematic reviews: the effects of two quality rating instruments on evidence regarding t'ai chi and bone mineral density in postmenopausal women

Abstract

Purpose: This article compares and contrasts two systematic reviews of t'ai chi (TC) interventions on bone mineral density in postmenopausal women. The aim is to examine how chosen quality rating instruments can impact systematic reviews of TC literature.

Methods: The rating instruments in the reviews, the three-item scale of Jadad et al. and the ad hoc checklist of Wayne et al., were analyzed using Oxman's evaluation criteria for systematic reviews regarding inclusion of articles, interpretation of results, and overall implications for the efficacy of TC on bone mineral density.

Results: According to Oxman's criteria, the Jadad scale did not address advances in statistical methods and was not comprehensive enough to adapt to the clinical context or topic. In contrast, the checklist by Wayne et al. was comprehensive, adaptable to clinical context and topical relevance, and compatible with recent developments in statistics and experimental design. These quality rating instruments were critical in the inclusion of studies, analyses, and overall conclusions summarizing the TC literature. The conclusions from the two systematic reviews were starkly opposing; Lee et al. found no convincing evidence, dismissing TC studies as low quality, while Wayne et al. stated that TC may be an effective, safe, and practical intervention.

Conclusions: Readers must exercise caution concerning high or low ratings from systematic reviews of TC studies because the choice of quality rating tool can dramatically influence the summary and conclusions of the reviews. There is no consensus on quality rating standards at this time. Of the two, the Jadad scale was not only inadequate but also inappropriate for reviewing TC studies, potentially misleading researchers, clinicians and policymakers. Future systematic reviews of TC should utilize instruments that are updated to current scientific standards, comprehensive, adaptable to clinical context, and relevant to the research topic.