Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jun 15;22(1):171.
doi: 10.1186/s12874-022-01650-5.

Trial-level characteristics associate with treatment effect estimates: a systematic review of meta-epidemiological studies

Huan Wang #  1 , Jinlu Song #  1 , Yali Lin  1 , Wenjie Dai  1 , Yinyan Gao  1 , Lang Qin  1 , Yancong Chen  1 , Wilson Tam  2 , Irene Xy Wu  3   4 , Vincent Ch Chung  5   6
Affiliations

Trial-level characteristics associate with treatment effect estimates: a systematic review of meta-epidemiological studies

Huan Wang et al. BMC Med Res Methodol. .

Abstract

Background: To summarize the up-to-date empirical evidence on trial-level characteristics of randomized controlled trials associated with treatment effect estimates.

Methods: A systematic review searched three databases up to August 2020. Meta-epidemiological (ME) studies of randomized controlled trials on intervention effect were eligible. We assessed the methodological quality of ME studies using a self-developed criterion. Associations between treatment effect estimates and trial-level characteristics were presented using forest plots.

Results: Eighty ME studies were included, with 25/80 (31%) being published after 2015. Less than one-third ME studies critically appraised the included studies (26/80, 33%), published a protocol (23/80, 29%), and provided a list of excluded studies with justifications (12/80, 15%). Trials with high or unclear (versus low) risk of bias on sequence generation (3/14 for binary outcome and 1/6 for continuous outcome), allocation concealment (11/18 and 1/6), double blinding (5/15 and 2/4) and smaller sample size (4/5 and 2/2) significantly associated with larger treatment effect estimates. Associations between high or unclear risk of bias on allocation concealment (5/6 for binary outcome and 1/3 for continuous outcome), double blinding (4/5 and 1/3) and larger treatment effect estimates were more frequently observed for subjective outcomes. The associations between treatment effect estimates and non-blinding of outcome assessors were removed in trials using multiple observers to reach consensus for both binary and continuous outcomes. Some trial characteristics in the Cochrane risk-of-bias (RoB2) tool have not been covered by the included ME studies, including using validated method for outcome measures and selection of the reported results from multiple outcome measures or multiple analysis based on results (e.g., significance of the results).

Conclusions: Consistently significant associations between larger treatment effect estimates and high or unclear risk of bias on sequence generation, allocation concealment, double blinding and smaller sample size were found. The impact of allocation concealment and double blinding were more consistent for subjective outcomes. The methodological and reporting quality of included ME studies were dissatisfactory. Future ME studies should follow the corresponding reporting guideline. Specific guidelines for conducting and critically appraising ME studies are needed.

Keywords: Meta-epidemiological study; Randomized controlled trial; Systematic review; Treatment effect estimates; Trial-level characteristic.

PubMed Disclaimer

Conflict of interest statement

None.

Figures

Fig. 1
Fig. 1
PRISMA flowchart: the literature search and selection of meta-epidemiological study on trial-level characteristics related to treatment effect estimates. ME, meta-epidemiological; RCT, randomized controlled trial
Fig. 2
Fig. 2
Associations between treatment effect estimates and trial-level characteristics for binary outcome
Fig. 3
Fig. 3
Associations between treatment effect estimates and trial-level characteristics for continuous outcome
Fig. 4
Fig. 4
Associations between treatment effect estimates and trial-level characteristics based on type of outcome (objective and subjective outcome). a binary outcome; b continuous outcome
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Moustgaard H, Clayton GL, Jones HE, Boutron I, Jørgensen L, Laursen DRT, Olsen MF, Paludan-Müller A, Ravaud P, Savović J, et al. Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study. BMJ (Clinical research ed) 2020;368:l6802. - PMC - PubMed
    1. Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ (Clinical research ed) 2012;344:e1119. doi: 10.1136/bmj.e1119. - DOI - PubMed
    1. Dechartres A, Ravaud P, Atal I, Riveros C, Boutron I: Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med. 2016;14(1):1–9. - PMC - PubMed
    1. Papageorgiou SN, Antonoglou GN, Tsiranidou E, Jepsen S, Jäger A. Bias and small-study effects influence treatment effect estimates: a meta-epidemiological study in oral medicine. J Clin Epidemiol. 2014;67(9):984–992. doi: 10.1016/j.jclinepi.2014.04.002. - DOI - PubMed
    1. Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155(1):39–51. doi: 10.7326/0003-4819-155-1-201107050-00006. - DOI - PubMed

Publication types

LinkOut - more resources