Factors affecting the comparability of meta-analyses and largest trials results in perinatology

Abstract

The objective of this report is to provide a new methodology for evaluating the performance of meta-analysis (MA) in corroborating results of large trials (LT) and to identify factors that could explain lack of similarity in the results. We used two criteria to judge the degree of similarity between a MA and the LT: (a) the ratio of the relative risk of the MA to the relative risk of the LT; and (b) the 95% confidence interval about this ratio. Furthermore, this degree of similarity was cross-tabulated with the presence or not of evidence of selective inclusion of positive studies (e.g., publication bias) as judged from "funnel plots" and statistical indicators. Depending on which of our two criteria was used, we found that between 20% and 53% of the 30 MAs studied have high or very high degree of similarity with the LT. We also found strong evidence that factors influencing asymmetrical funnel plots of MA, such as publication bias, may play an important role in this degree of similarity. There was a sizeable proportion of meta-analyses that did not agree with large trial results. We recommend that funnel plots be used as a tool for identifying which MAs can mislead. However, the statistical indicators at hand are unlikely to be of use in many area of medicine considering the regrettably small number of randomized controlled trials per topic available.