Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct 11;20(1):330.
doi: 10.1186/s12916-022-02540-9.

Empirical evidence of study design biases in nutrition randomised controlled trials: a meta-epidemiological study

Julia Stadelmaier  1 , Isabelle Roux  2 , Maria Petropoulou  3 , Lukas Schwingshackl  2
Affiliations

Empirical evidence of study design biases in nutrition randomised controlled trials: a meta-epidemiological study

Julia Stadelmaier et al. BMC Med. .

Abstract

Background: Instruments to critically appraise randomised controlled trials (RCTs) are based on evidence from meta-epidemiological studies. We aim to conduct a meta-epidemiological study on the average bias associated with reported methodological trial characteristics such as random sequence generation, allocation concealment, blinding, incomplete outcome data, selective reporting, and compliance of RCTs in nutrition research.

Methods: We searched the Cochrane Database of Systematic Reviews, for systematic reviews of RCTs, published between 01 January 2010 and 31 December 2019. We combined the estimates of the average bias (e.g. ratio of risk ratios [RRR] or differences in standardised mean differences) in meta-analyses using the random-effects model. Subgroup analyses were conducted to investigate the potential differences among the RCTs with low versus high/unclear risk of bias with respect to the different types of interventions (e.g. micronutrients, fatty acids, dietary approach), outcomes (e.g. mortality, pregnancy outcomes), and type of outcome (objective, subjective). Heterogeneity was assessed through I2 and τ2, and prediction intervals were calculated.

Results: We included 27 Cochrane nutrition reviews with 77 meta-analyses (n = 927 RCTs). The available evidence suggests that intervention effect estimates may not be exaggerated in RCTs with high/unclear risk of bias (versus low) judgement for sequence generation (RRR 0.97, 95% CI 0.93 to 1.02; I2 = 28%; τ2 = 0.002), allocation concealment (RRR 1.00, 95% CI 0.96 to 1.04; I2 = 27%; τ2 = 0.001), blinding of participants and personnel (RRR 0.95, 95% CI 0.91 to 1.00; I2 = 23%; τ2 = 0), selective reporting (RRR 0.97, 95% CI 0.92 to 1.02; I2 = 24%; τ2 = 0), and compliance (RRR 0.95, 95% CI 0.89 to 1.02; I2 = 0%; τ2 = 0). Intervention effect estimates seemed to be exaggerated in RCTs with a high/unclear risk of bias judgement for blinding of outcome assessment (RRR 0.81, 95% CI 0.70 to 0.94; I2 = 26%; τ2 = 0.03), which was predominately driven by subjective outcomes, and incomplete outcome data (RRR 0.92, 95% CI 0.88 to 0.97; I2 = 22%; τ2 = 0.001). For continuous outcomes, no differences were observed, except for selective reporting.

Conclusions: On average, most characteristics of nutrition RCTs may not exaggerate intervention effect estimates, but the average bias appears to be greatest in trials of subjective outcomes. Replication of this study is suggested in this field to keep this conclusion updated.

Keywords: Cohort studies; Dietary compliance; Meta-analysis; Nutrition; Pooling; Randomised controlled trials; Risk of bias.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Flow diagram showing study selection process for eligible Cochrane reviews. MA, meta-analyses; RCT, randomised controlled trials; SR, systematic reviews. Reasons for exclusion are displayed in Additional file 3: Tables S8-S9
See this image and copyright information in PMC

References

    1. Afshin A, Sur PJ, Fay KA, Cornaby L, Ferrara G, Salama JS, et al. Health effects of dietary risks in 195 countries, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet. 2019;393(10184):1958–1972. doi: 10.1016/S0140-6736(19)30041-8. - DOI - PMC - PubMed
    1. Johnston BC, Seivenpiper JL, Vernooij RWM, de Souza RJ, Jenkins DJA, Zeraatkar D, et al. The philosophy of evidence-based principles and practice in nutrition. Mayo Clin Proc Innov Qual Outcomes. 2019;3(2):189–199. doi: 10.1016/j.mayocpiqo.2019.02.005. - DOI - PMC - PubMed
    1. Lichtenstein AH, Petersen K, Barger K, Hansen KE, Anderson CAM, Baer DJ, et al. Perspective: design and conduct of human nutrition randomized controlled trials. Adv Nutr. 2021;12(1):4–20. doi: 10.1093/advances/nmaa109. - DOI - PMC - PubMed
    1. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–605. doi: 10.1136/bmj.39465.451748.AD. - DOI - PMC - PubMed
    1. Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. Chapter 7: Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane. 2022. Available from www.training.cochrane.org/handbook.

Publication types

LinkOut - more resources