Capture-recapture is a potentially useful method for assessing publication bias

Abstract

Objectives: Publication bias is a problem in systematic reviews of randomized controlled trials. The aim of this study was to assess the influence of publication bias in a systematic review of the effectiveness of Progressive Resistance Training (PRT) in older people.

Study design and setting: The relevant studies were ascertained from three sources: electronic databases, experts, and handsearching. Capture-recapture, visual inspection of funnel plots, two statistical tests, and two methods that make adjustments for publication bias, were employed to check the robustness of the conclusions of the systematic review.

Results: The methods employed gave broadly consistent results. Capture-recapture estimated that 3 (95% CI ) relevant studies were missed, while Trim and Fill suggested 16 studies had been missed. Both Egger's test for bias and a funnel plot regression approach suggested that publication bias was present. A selection model approach suggested that the funnel plot asymmetry observed may not be entirely due to publication bias.

Conclusion: Capture-recapture is a potentially useful method for assessing publication bias. Further research in the form of simulation studies is required, using a variety of scenarios to investigate the extent to which each method approximates the truth.