Inter-study differences: how should they influence the interpretation and analysis of results?

Abstract

In determining the role inter-study variation should play in an overview analysis, it is important to consider three factors: which question one is trying to answer; the degree of similarity or dissimilarity of design, and the degree to which heterogeneity of outcomes can be explained. Three questions one might be interested in are: whether treatment can be effective in some circumstances; whether treatment is effective on average, and whether treatment was effective on average in the trials at hand. Under the assumption of no qualitative interaction, the answers to these questions coincide. The O-E analysis most directly answers the third question. Other analyses are suggested when the first question is of interest, using the aspirin post-MI studies as an example.