Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

doi:10.1093/ajcn/nqab002

. 2021 Jun 1;113(6):1578-1592.

doi: 10.1093/ajcn/nqab002.

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

Dena Zeraatkar^{1

2}, Arrti Bhasin¹, Rita E Morassut³, Isabella Churchill¹, Arnav Gupta⁴, Daeria O Lawson¹, Anna Miroshnychenko¹, Emily Sirotich¹, Komal Aryal¹, David Mikhail⁵, Tauseef A Khan^{6

7}, Vanessa Ha⁸, John L Sievenpiper^{6

7}, Steven E Hanna¹, Joseph Beyene¹, Russell J de Souza^{1

7

9}

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
² Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
³ Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada.
⁴ Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁵ Faculty of Science, McMaster University, Hamilton, Ontario, Canada.
⁶ Department of Nutritional Sciences, Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
⁷ 3D Knowledge Synthesis and Clinical Trials Unit, Clinical Nutrition and Risk Factor Modification Centre, Division of Endocrinology & Metabolism, St. Michael's Hospital, Toronto, Ontario, Canada.
⁸ School of Medicine, Queen's University, Kingston, Ontario, Canada.
⁹ Population Health Research Institute, McMaster University, Hamilton, Ontario, Canada.

PMID: 33740039
PMCID: PMC8243916
DOI: 10.1093/ajcn/nqab002

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

Dena Zeraatkar et al. Am J Clin Nutr. 2021.

. 2021 Jun 1;113(6):1578-1592.

doi: 10.1093/ajcn/nqab002.

Authors

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
² Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
³ Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada.
⁴ Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁵ Faculty of Science, McMaster University, Hamilton, Ontario, Canada.
⁶ Department of Nutritional Sciences, Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
⁷ 3D Knowledge Synthesis and Clinical Trials Unit, Clinical Nutrition and Risk Factor Modification Centre, Division of Endocrinology & Metabolism, St. Michael's Hospital, Toronto, Ontario, Canada.
⁸ School of Medicine, Queen's University, Kingston, Ontario, Canada.
⁹ Population Health Research Institute, McMaster University, Hamilton, Ontario, Canada.

PMID: 33740039
PMCID: PMC8243916
DOI: 10.1093/ajcn/nqab002

Abstract

Background: Dietary recommendations and policies should be guided by rigorous systematic reviews. Reviews that are of poor methodological quality may be ineffective or misleading. Most of the evidence in nutrition comes from nonrandomized studies of nutritional exposures (usually referred to as nutritional epidemiology studies), but to date methodological evaluations of the quality of systematic reviews of such studies have been sparse and inconsistent.

Objectives: We aimed to investigate the quality of recently published systematic reviews and meta-analyses of nutritional epidemiology studies and to propose guidance addressing major limitations.

Methods: We searched MEDLINE (January 2018-August 2019), EMBASE (January 2018-August 2019), and the Cochrane Database of Systematic Reviews (January 2018-February 2019) for systematic reviews of nutritional epidemiology studies. We included a random sample of 150 reviews.

Results: Most reviews were published by authors from Asia (n = 49; 32.7%) or Europe (n = 43; 28.7%) and investigated foods or beverages (n = 60; 40.0%) and cancer morbidity and mortality (n = 54; 36%). Reviews often had important limitations: less than one-quarter (n = 30; 20.0%) reported preregistration of a protocol and almost one-third (n = 42; 28.0%) did not report a replicable search strategy. Suboptimal practices and errors in the synthesis of results were common: one-quarter of meta-analyses (n = 30; 26.1%) selected the meta-analytic model based on statistical indicators of heterogeneity and almost half of meta-analyses (n = 50; 43.5%) did not consider dose-response associations even when it was appropriate to do so. Only 16 (10.7%) reviews used an established system to evaluate the certainty of evidence.

Conclusions: Systematic reviews of nutritional epidemiology studies often have serious limitations. Authors can improve future reviews by involving statisticians, methodologists, and researchers with substantive knowledge in the specific area of nutrition being studied and using a rigorous and transparent system to evaluate the certainty of evidence.

Keywords: credibility; nutritional epidemiology; quality; risk of bias; systematic reviews.

PubMed Disclaimer

Comment in

Grading nutrition evidence: where to go from here?
Tobias DK, Wittenbecher C, Hu FB. Tobias DK, et al. Am J Clin Nutr. 2021 Jun 1;113(6):1385-1387. doi: 10.1093/ajcn/nqab124. Am J Clin Nutr. 2021. PMID: 33963733 Free PMC article. No abstract available.

Cited by

Assessments of risk of bias in systematic reviews of observational nutritional epidemiologic studies are often not appropriate or comprehensive: a methodological study.
Zeraatkar D, Kohut A, Bhasin A, Morassut RE, Churchill I, Gupta A, Lawson D, Miroshnychenko A, Sirotich E, Aryal K, Azab M, Beyene J, de Souza RJ. Zeraatkar D, et al. BMJ Nutr Prev Health. 2021 Dec 7;4(2):487-500. doi: 10.1136/bmjnph-2021-000248. eCollection 2021. BMJ Nutr Prev Health. 2021. PMID: 35028518 Free PMC article.
NUQUEST-NUtrition QUality Evaluation Strengthening Tools: development of tools for the evaluation of risk of bias in nutrition studies.
Kelly SE, Greene-Finestone LS, Yetley EA, Benkhedda K, Brooks SPJ, Wells GA, MacFarlane AJ. Kelly SE, et al. Am J Clin Nutr. 2022 Jan 11;115(1):256-271. doi: 10.1093/ajcn/nqab335. Am J Clin Nutr. 2022. PMID: 34605544 Free PMC article.
Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention.
Storman D, Koperny M, Zając J, Polak M, Weglarz P, Bochenek-Cibor J, Swierz MJ, Staskiewicz W, Gorecka M, Skuza A, Wach AA, Kaluzinska K, Bała MM. Storman D, et al. Int J Environ Res Public Health. 2022 Jan 3;19(1):506. doi: 10.3390/ijerph19010506. Int J Environ Res Public Health. 2022. PMID: 35010766 Free PMC article.
Grilling the data: application of specification curve analysis to red meat and all-cause mortality.
Wang Y, Pitre T, Wallach JD, de Souza RJ, Jassal T, Bier D, Patel CJ, Zeraatkar D. Wang Y, et al. J Clin Epidemiol. 2024 Apr;168:111278. doi: 10.1016/j.jclinepi.2024.111278. Epub 2024 Feb 12. J Clin Epidemiol. 2024. PMID: 38354868 Free PMC article.
Nuts and seeds consumption and risk of cardiovascular disease, type 2 diabetes and their risk factors: a systematic review and meta-analysis.
Arnesen EK, Thorisdottir B, Bärebring L, Söderlund F, Nwaru BI, Spielau U, Dierkes J, Ramel A, Lamberg-Allardt C, Åkesson A. Arnesen EK, et al. Food Nutr Res. 2023 Feb 14;67. doi: 10.29219/fnr.v67.8961. eCollection 2023. Food Nutr Res. 2023. PMID: 36816545 Free PMC article. Review.

See all "Cited by" articles

References

1. Ortiz-Moncada R, González-Zapata L, Ruiz-Cantero MT, Clemente-Gómez V. Priority issues, study designs and geographical distribution in nutrition journals. Nutr Hosp. 2011;26(4):784–91. - PubMed
1. Zeraatkar D, Johnston BC, Guyatt G. Evidence collection and evaluation for the development of dietary guidelines and public policy on nutrition. Annu Rev Nutr. 2019;39:227–47. - PubMed
1. Hebert JR, Frongillo EA, Adams SA, Turner-McGrievy GM, Hurley TG, Miller DR, Ockene IS. Perspective: randomized controlled trials are not a panacea for diet-related research. Adv Nutr. 2016;7(3):423–32. - PMC - PubMed
1. Brannon PM, Taylor CL, Coates PM. Use and applications of systematic reviews in public health nutrition. Annu Rev Nutr. 2014;34:401–19. - PubMed
1. Barnard ND, Willett WC, Ding EL. The misuse of meta-analysis in nutrition research. JAMA. 2017;318(15):1435–6. - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

CIHR/Canada

LinkOut - more resources

Full Text Sources
Other Literature Sources
- H1 Connect - Access expert opinions and insights on biomedical research.
- scite Smart Citations

[1] Ortiz-Moncada R, González-Zapata L, Ruiz-Cantero MT, Clemente-Gómez V. Priority issues, study designs and geographical distribution in nutrition journals. Nutr Hosp. 2011;26(4):784–91. - PubMed

[2] Ortiz-Moncada R, González-Zapata L, Ruiz-Cantero MT, Clemente-Gómez V. Priority issues, study designs and geographical distribution in nutrition journals. Nutr Hosp. 2011;26(4):784–91. - PubMed

[3] Zeraatkar D, Johnston BC, Guyatt G. Evidence collection and evaluation for the development of dietary guidelines and public policy on nutrition. Annu Rev Nutr. 2019;39:227–47. - PubMed

[4] Zeraatkar D, Johnston BC, Guyatt G. Evidence collection and evaluation for the development of dietary guidelines and public policy on nutrition. Annu Rev Nutr. 2019;39:227–47. - PubMed

[5] Hebert JR, Frongillo EA, Adams SA, Turner-McGrievy GM, Hurley TG, Miller DR, Ockene IS. Perspective: randomized controlled trials are not a panacea for diet-related research. Adv Nutr. 2016;7(3):423–32. - PMC - PubMed

[6] Hebert JR, Frongillo EA, Adams SA, Turner-McGrievy GM, Hurley TG, Miller DR, Ockene IS. Perspective: randomized controlled trials are not a panacea for diet-related research. Adv Nutr. 2016;7(3):423–32. - PMC - PubMed

[7] Brannon PM, Taylor CL, Coates PM. Use and applications of systematic reviews in public health nutrition. Annu Rev Nutr. 2014;34:401–19. - PubMed

[8] Brannon PM, Taylor CL, Coates PM. Use and applications of systematic reviews in public health nutrition. Annu Rev Nutr. 2014;34:401–19. - PubMed

[9] Barnard ND, Willett WC, Ding EL. The misuse of meta-analysis in nutrition research. JAMA. 2017;318(15):1435–6. - PubMed

[10] Barnard ND, Willett WC, Ding EL. The misuse of meta-analysis in nutrition research. JAMA. 2017;318(15):1435–6. - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

Affiliations

Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study

Authors

Affiliations

Abstract

Comment in

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources