Empirical evaluation showed that the Copas selection model provided a useful summary in 80% of meta-analyses

Abstract

Objective: Although using meta-analysis to combine evidence from a number of studies should reduce both bias and uncertainty, it is sometimes not the case, because published studies represent a biased selection of the evidence. Copas proposed a selection model to assess the sensitivity of meta-analysis conclusions to possible selection bias. However, this relatively complex model awaits both reliable software and an empirical evaluation. This article reports work addressing both these issues.

Study design and setting: We took 157 meta-analyses with binary outcomes, analyzed each one using the Copas selection model, and evaluated each analysis using a prespecified protocol. The evaluation aimed to assess the usefulness of the Copas selection model to a typical Cochrane reviewer.

Results: In approximately 80% of meta-analyses, the overall interpretation of the Copas selection model was clear, with better results among the 22 with evidence of selection bias. However, as with the "Trim and Fill" method, allowing for selection bias can result in smaller standard errors for the treatment estimate.

Conclusion: When a reliable test for selection bias is significant, we recommend systematic reviewers to try the Copas selection model, although the results should be interpreted cautiously.