Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis

Abstract

Background and objective: Cumulative meta-analyses are prone to produce spurious P<0.05 because of repeated testing of significance as trial data accumulate. Information size in a meta-analysis should at least equal the sample size of an adequately powered trial. Trial sequential analysis (TSA) corresponds to group sequential analysis of a single trial and may be applied to meta-analysis to evaluate the evidence.

Study design and setting: Six randomly selected neonatal meta-analyses with at least five trials reporting a binary outcome were examined. Low-bias heterogeneity-adjusted information size and information size determined from an assumed intervention effect of 15% were calculated. These were used for constructing trial sequential monitoring boundaries. We assessed the cumulative z-curves' crossing of P=0.05 and the boundaries.

Results: Five meta-analyses showed early potentially spurious P<0.05 values. In three significant meta-analyses the cumulative z-curves crossed both boundaries, establishing firm evidence of an intervention effect. In two nonsignificant meta-analyses the cumulative z-curves crossed P=0.05, but never the boundaries, demonstrating early potentially spurious P<0.05 values. In one nonsignificant meta-analysis the cumulative z-curves never crossed P=0.05 or the boundaries.

Conclusion: TSAs may establish when firm evidence is reached in meta-analysis.