Reporting of Protocol Rationale and Content Validity in Randomized Clinical Trials of T'ai Chi: A Systematic Evaluation

Abstract

Background: Mind-body exercise interventions are typically multimodal, complex, and pluralistic, and few have been developed with the goal of therapeutically targeting a specific medical population. It is thus important that clinical trials evaluating mind-body interventions provide some justification for the use of the specific protocol being evaluated.

Objectives: This article reports the results of a systematic review of the quality of reporting of protocol rationale and content validity for using a specific t'ai chi protocol in a randomized controlled trial (RCT).

Methods: Electronic literature searches were conducted using PubMed/MEDLINE, EBSCOhost, and the Cochrane Library from inception through June 2015. Search terms were Tai Chi, Taiji, Tai Chi Chuan; searches were limited to English-language RCTs. Inclusion and exclusion of trials were reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The authors assessed the level of reporting with respect to t'ai chi protocol validation using a 5-point grading system based on whether (1) a specific protocol was mentioned, (2) rationale for the specific protocol was stated and supported, and (3) formal evaluation for content validity was conducted.

Results: There was significant heterogeneity in the quality of reporting related to protocol rationale and content validity. A total of 171 publications were identified. Studies met between 0 and 4 validity criteria (of possible 5, more indicating better quality), with a mean of 2.52 (±SD 1.2) and median of 3. Twenty (12%) trials did not mention a specific t'ai chi protocol, 10 (6%) trials met 0 of 5 criteria, and 47 (31%) studies met 4 of 5 criteria. Formal validity assessments were employed in only one trial.

Conclusions: The poor quality of protocol rationale and content validity reporting limits our ability to accurately evaluate the evidence of t'ai chi as a therapeutic intervention. The development of formal guidelines for developing and reporting intervention validity for multimodal mind-body exercises like t'ai chi may improve the quality and interpretability of research.

Keywords: content validity; methodology; mind–body; nonpharmacological intervention; protocol rationale.

FIG. 1.

Scoring criteria flowchart. Specific t'ai chi protocol: was a specific t'ai chi protocol mentioned as opposed to t'ai chi in general? For example, Yang style 24. Therapeutic principles/components of specific t'ai chi protocol: were therapeutic components of the specific t'ai chi protocol mentioned or described? For example, breathing, mindfulness, meditation, relaxation, imagery and visualization, biomechanical integration, and physical exercise. Population/outcomes matched to the specific t'ai chi protocol: was there explicit mention of the relevancy or appropriateness of the specific t'ai chi protocol and/or its therapeutic principles to the study population or study outcomes? References: were references provided that support the use of the specific t'ai chi protocol? Formal content validity testing: was there a formal evaluation of the content validity of the t'ai chi protocol? For example, a Delphi process.

FIG. 2.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomized controlled trial.

FIG. 3.

Characteristics of t'ai chi protocols used in RCTs. (a) Percentage of studies employing each of the five different types of t'ai chi protocols. (b) Percentage of studies meeting specific validation criteria. RCT, randomized controlled trial.