Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2021 Apr 21:2021:9975351.
doi: 10.1155/2021/9975351. eCollection 2021.

The Use of Multiple Primary Outcomes in Randomized Controlled Trials of Chinese Herbal Medicine

Jing Hu  1 , Shuo Feng  1 , Xiaoli Zhang  2 , Huina Zhang  1 , Yanxiang Ha  1 , Chongyang Wei  1 , Xuejiao Wang  1 , Rui Zhang  1 , Xing Liao  3 , Bo Li  1
Affiliations
Review

The Use of Multiple Primary Outcomes in Randomized Controlled Trials of Chinese Herbal Medicine

Jing Hu et al. Evid Based Complement Alternat Med. .

Abstract

Background: Multiple primary outcomes are commonly used in randomized controlled trials (RCTs) of Chinese herbal medicine (CHM). Analysis and interpretation of the results of CHM RCTs with many outcomes are not clear. No previous studies have systematically assessed the use of multiple primary outcomes in this area. This study aimed to assess the reporting of multiple primary outcomes and the statistical methods used to adjust multiplicity in RCTs of CHM.

Methods: Search for RCTs of CHM published in English between January 2010 and December 2019 in MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials (CENTRAL) was undertaken. We randomly selected 20% of the included RCTs as the analyzing sample of this study. The number of multiple primary outcomes, the methods used to adjust the multiplicity in statistical analysis and sample size estimate, and the trial information were collected. For RCTs that adopted multiple primary outcomes without the multiplicity adjustment, we used Bonferroni correction to adjust.

Results: 227 CHM RCTs were included in our study. 92 (40.5%) failed to report what their primary outcome was. Of 135 (59.5%) RCTs that reported primary outcome, 93 (68.9%) reported one and 42 (31.1%) reported more than one primary outcome (range 2-5). Of 42 RCTs that reported multiple primary outcomes, only 5 adjusted for multiple outcomes. If multiplicity had been accounted for using Bonferroni correction, 10 (37.0%) RCTs that reported a significant result had demonstrated a nonsignificant result, giving the adjusted P value. Only one of the 42 RCTs calculated sample size based on multiple primary outcomes. Adopting multiple primary outcomes showed a slow growth trend with the publication year. The proportion of primary outcome reported explicitly in RCTs was different in terms of the nationality of the first author (P=0.004), in which mainland China has the lowest proportion (55.8%). The highest percentage of the studies with primary outcome reporting explicitation was mental and behavioural disorders (83.3%), and the most frequently adopting multiple primary outcomes were studies on the disease of the nervous system (66.7%). The percentage of reporting primary outcome explicitly was associated with sample size (P < 0.001); for the percentage of RCTs adopting multiple primary outcomes, there was no statistically significant difference (P=0.739).

Conclusions: Multiple primary outcomes are prevalent in CHM RCTs. However, appropriate methods are not usually taken in most of the analyses to safeguard the inferences against multiplicity. Sample size estimation based on multiple primary outcomes is still lacking. These issues complicate the interpretability of trial results and can lead to spurious conclusions. Guidelines to improve analyzing and reporting for multiple primary outcomes in CHM RCTs are warranted.

PubMed Disclaimer

Conflict of interest statement

The authors declare that there are no conflicts of interest regarding the publication of this article.

Figures

Figure 1
Figure 1
Flow chart of study selection.
Figure 2
Figure 2
Number of reported outcomes in 227 included RCTs.
Figure 3
Figure 3
Percentage of RCTs that reported primary outcome and primary multiple outcomes by publication year.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Wang J., Guo Y., Li G. Current status of standardization of traditional Chinese medicine in China. Evidence-Based Complementary and Alternative Medicine. 2016;2016:7. doi: 10.1155/2016/9123103.9123103 - DOI - PMC - PubMed
    1. Lin A., Chan G., Hu Y., et al. Internationalization of traditional Chinese medicine: current international market, internationalization challenges and prospective suggestions. Chinese Medicine. 2018;13:p. 9. doi: 10.1186/s13020-018-0167-z. - DOI - PMC - PubMed
    1. Rashrash M., Schommer J. C., Brown L. M. Prevalence and predictors of herbal medicine use among adults in the United States. Journal of Patient Experience. 2017;4(3):108–113. doi: 10.1177/2374373517706612. - DOI - PMC - PubMed
    1. Wang W.-J., Zhang T. Integration of traditional Chinese medicine and Western medicine in the era of precision medicine. Journal of Integrative Medicine. 2017;15(1):1–7. doi: 10.1016/s2095-4964(17)60314-5. - DOI - PubMed
    1. Chen K., Qian Z., Zhang W., et al. Analysis of the effect of refined Guanxin tablets on 112 patients with angina pectoris. Bulletin of Medical Research. 1982;11:24–25.

LinkOut - more resources