Pooling of cohort studies and RCTs affects GRADE certainty of evidence in nutrition research

Abstract

Background: There is a little empirical evidence of the impact of pooling randomized controlled trials (RCTs) and cohort studies (CSs) on the certainty-of-evidence. To evaluate the hypothetical-scenario of pooling bodies-of-evidence from RCTs with matched bodies-of-evidence from CSs on the certainty-of-evidence.

Methods: We extracted GRADE ratings of bodies-of-evidence from RCTs in Cochrane reviews, and rated the certainty-of-evidence from matched bodies-of-evidence from CSs. We then evaluated the impact of pooling both bodies-of-evidence on the overall certainty-of-evidence, and on individual GRADE domains.

Results: Fourty-two pooled bodies-of-evidence were rated, ranging from very-low (bodies-of-evidence_RCTs: 9.5%; bodies-of-evidence_CSs: 40.5%; pooled-bodies-of-evidence: 0%) to low (bodies-of-evidence_RCTs: 38.1%; bodies-of-evidence_CSs: 45.2%; pooled-bodies-of-evidence: 19.1%), moderate (bodies-of-evidence_RCTs: 33.4%; bodies-of-evidence_CSs: 14.3%; pooled-bodies-of-evidence: 57.1%), and high (bodies-of-evidence_RCTs: 19%; bodies-of-evidence_CSs: 0%; pooled-bodies-of-evidence: 23.8%). Certainty-of-evidence was downgraded mostly for imprecision and risk of bias for bodies-of-evidence from RCTs, and for risk of bias and inconsistency for bodies-of-evidence from CSs. Pooling both bodies-of-evidence mitigates rating down for imprecision compared to bodies-of-evidence from RCTs and inconsistency compared to bodies-of-evidence from CSs.

Conclusion: Our hypothetical study suggests that pooling both bodies-of-evidence would reduce the amount of very-low and low certainty-of-evidence ratings, but how to integrate RCTs and CSs and whether or not to pool these bodies-of-evidence requires proper guidance before systematic review authors or guideline developers should consider this approach.

Keywords: Certainty of evidence; Cohort studies; GRADE; Nutrition; Pooling; RCTs.